-- Logs begin at Wed 2022-06-15 13:47:11 UTC, end at Wed 2022-06-15 15:12:07 UTC. -- Jun 15 14:27:55.469020 dv-cinder-803823-10-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 15 14:27:55.476326 dv-cinder-803823-10-iscsi devstack@keystone.service[80472]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 15 14:27:55.478367 dv-cinder-803823-10-iscsi devstack@keystone.service[80472]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jun 15 14:27:55 2022] *** Jun 15 14:27:55.478367 dv-cinder-803823-10-iscsi devstack@keystone.service[80472]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 15 14:27:55.478367 dv-cinder-803823-10-iscsi devstack@keystone.service[80472]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 15 14:27:55.478367 dv-cinder-803823-10-iscsi devstack@keystone.service[80472]: nodename: dv-cinder-803823-10-iscsi Jun 15 14:27:55.478367 dv-cinder-803823-10-iscsi devstack@keystone.service[80472]: machine: x86_64 Jun 15 14:27:55.478367 dv-cinder-803823-10-iscsi devstack@keystone.service[80472]: clock source: unix Jun 15 14:27:55.478367 dv-cinder-803823-10-iscsi devstack@keystone.service[80472]: pcre jit disabled Jun 15 14:27:55.478367 dv-cinder-803823-10-iscsi devstack@keystone.service[80472]: detected number of CPU cores: 4 Jun 15 14:27:55.478367 dv-cinder-803823-10-iscsi devstack@keystone.service[80472]: current working directory: / Jun 15 14:27:55.478933 dv-cinder-803823-10-iscsi devstack@keystone.service[80472]: detected binary path: /usr/bin/uwsgi-core Jun 15 14:27:55.478933 dv-cinder-803823-10-iscsi devstack@keystone.service[80472]: your processes number limit is 47680 Jun 15 14:27:55.478933 dv-cinder-803823-10-iscsi devstack@keystone.service[80472]: your memory page size is 4096 bytes Jun 15 14:27:55.478933 dv-cinder-803823-10-iscsi devstack@keystone.service[80472]: detected max file descriptor number: 2048 Jun 15 14:27:55.478933 dv-cinder-803823-10-iscsi devstack@keystone.service[80472]: lock engine: pthread robust mutexes Jun 15 14:27:55.478933 dv-cinder-803823-10-iscsi devstack@keystone.service[80472]: thunder lock: enabled Jun 15 14:27:55.478933 dv-cinder-803823-10-iscsi devstack@keystone.service[80472]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 15 14:27:55.478933 dv-cinder-803823-10-iscsi devstack@keystone.service[80472]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 15 14:27:55.524200 dv-cinder-803823-10-iscsi devstack@keystone.service[80472]: Python main interpreter initialized at 0x558a01f4c9c0 Jun 15 14:27:55.524200 dv-cinder-803823-10-iscsi devstack@keystone.service[80472]: python threads support enabled Jun 15 14:27:55.524200 dv-cinder-803823-10-iscsi devstack@keystone.service[80472]: your server socket listen backlog is limited to 100 connections Jun 15 14:27:55.524200 dv-cinder-803823-10-iscsi devstack@keystone.service[80472]: your mercy for graceful operations on workers is 90 seconds Jun 15 14:27:55.524560 dv-cinder-803823-10-iscsi devstack@keystone.service[80472]: mapped 403077 bytes (393 KB) for 2 cores Jun 15 14:27:55.524661 dv-cinder-803823-10-iscsi devstack@keystone.service[80472]: *** Operational MODE: preforking *** Jun 15 14:27:55.524937 dv-cinder-803823-10-iscsi devstack@keystone.service[80472]: *** uWSGI is running in multiple interpreter mode *** Jun 15 14:27:55.524937 dv-cinder-803823-10-iscsi devstack@keystone.service[80472]: spawned uWSGI master process (pid: 80472) Jun 15 14:27:55.525138 dv-cinder-803823-10-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 15 14:27:55.526803 dv-cinder-803823-10-iscsi devstack@keystone.service[80472]: spawned uWSGI worker 1 (pid: 80481, cores: 1) Jun 15 14:27:55.526803 dv-cinder-803823-10-iscsi devstack@keystone.service[80472]: spawned uWSGI worker 2 (pid: 80482, cores: 1) Jun 15 14:27:55.526803 dv-cinder-803823-10-iscsi devstack@keystone.service[80472]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 15 14:28:01.078840 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 14:28:01.078840 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: from cryptography.utils import int_from_bytes Jun 15 14:28:01.079615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 14:28:01.079615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: from cryptography.utils import int_from_bytes Jun 15 14:28:01.079724 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 14:28:01.079724 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: from cryptography.utils import int_from_bytes Jun 15 14:28:01.080119 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 14:28:01.080119 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: from cryptography.utils import int_from_bytes Jun 15 14:28:01.129251 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 15 14:28:01.129445 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 15 14:28:01.129638 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] command line args: [] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 15 14:28:01.129922 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 15 14:28:01.130102 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ================================================================================ {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 15 14:28:01.130355 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] admin_token = **** {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.130598 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] config_dir = [] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.130785 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.130968 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] config_source = [] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.131172 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] debug = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.131374 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.131865 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.131950 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.132166 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.132413 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.132647 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] list_limit = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.132827 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] log_config_append = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.133005 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.133184 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] log_dir = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.133361 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] log_file = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.133606 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.133846 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.134102 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.134348 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.134526 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.134705 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.134948 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.135217 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.135388 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 15 14:28:01.135474 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.135614 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 15 14:28:01.135693 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.135812 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] command line args: [] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 15 14:28:01.135931 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.136094 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 15 14:28:01.136171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.136275 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ================================================================================ {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 15 14:28:01.136408 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.136532 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] admin_token = **** {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.136639 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.136781 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] config_dir = [] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.136913 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.136996 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.137147 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] config_source = [] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.137331 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] debug = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.137419 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.45/identity {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.137539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.137645 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] publish_errors = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.137781 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.137866 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.137963 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.138052 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.138169 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.138248 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.138408 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.138535 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.138645 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.138722 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] list_limit = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.138860 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] log_config_append = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.138936 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] standard_threads = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.139069 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.139149 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.139252 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] log_dir = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.139356 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.139434 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] log_file = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.139535 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.139659 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.139743 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] use_journal = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.139875 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] use_json = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.139957 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.140050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] use_stderr = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.140154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.140239 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] use_syslog = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.140403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.140481 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.140846 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.140950 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.141035 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.141192 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.141274 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.141452 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.141547 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.141698 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.141777 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.141945 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.142027 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.142182 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.142258 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.142414 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.142499 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.142650 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.142726 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.142896 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.142973 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.143168 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.143254 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.143417 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.143651 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.143732 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.45/identity {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.143849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] publish_errors = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.143932 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.144085 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.144169 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.144319 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.144403 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.144502 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.144579 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.144749 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.144826 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.144986 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.145065 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.145224 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] standard_threads = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.145308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.145457 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.145535 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.145715 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.145793 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.145896 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.145973 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.146075 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] use_journal = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.146226 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.146314 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] use_json = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.146477 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.146555 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] use_stderr = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.146673 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] use_syslog = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.146755 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.146910 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:01.147021 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.147186 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.147277 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.147417 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.147495 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.147651 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.147806 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.147885 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.147957 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.148132 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.148217 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.148362 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.148447 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.148594 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.148671 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.148840 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.148916 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.149085 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.149170 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.149322 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.149400 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.149570 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.149645 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.149797 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.149880 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.150032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.150116 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.150265 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.150342 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.150495 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.150583 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.150724 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.150807 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.150953 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.151058 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.151736 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.151831 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.151973 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.152049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.152224 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.152300 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.152455 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.152535 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.152691 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.152769 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.152918 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.152995 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.153151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.153235 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.153384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.153462 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.153629 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.153706 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.153849 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.153923 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.154081 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.154159 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.154317 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.154394 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.154551 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.154630 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.154786 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.154864 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.155039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.155120 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.155272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.155358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.155519 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.155629 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.155761 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.155864 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.155999 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.156104 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.156230 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.156351 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.156457 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.156540 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.156697 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.156776 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.156927 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.157005 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.157157 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.157266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.157400 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.157477 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.157593 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.157673 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.157831 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.157909 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.158070 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.158153 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.158321 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.158407 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.158562 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.158662 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.158790 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.158873 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.159033 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.159127 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.159276 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.159353 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.159513 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.159639 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.159757 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.159891 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.159979 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.160053 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.160215 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.160308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.160447 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.160533 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.160681 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.160755 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.160918 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.161008 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.161155 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.161246 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.161392 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.161469 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.161646 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.161731 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.161885 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.161963 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.162130 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.162210 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.162372 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.162460 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.162609 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.162693 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.162846 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.162929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.163103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.163198 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.163338 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.163415 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.163569 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.163644 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.163799 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.163883 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.164035 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.164112 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.164264 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.164340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.164505 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.164589 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.164812 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.165054 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.165293 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.165547 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.165789 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.166031 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.166199 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.166276 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.166477 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.166553 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.166773 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.167414 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.167577 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.167675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.167853 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.167941 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.168097 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.168175 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.168340 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] auth.external = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.168424 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.168582 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.168660 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.168849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.168929 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.169094 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.169173 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.169342 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] auth.password = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.169418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.169578 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] auth.token = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.169655 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.169816 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.169894 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.170054 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.170130 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.170286 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.170361 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.170522 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.170600 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.170756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.170832 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.171016 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.171096 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.171248 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.171358 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.171485 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] credential.caching = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.171571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.171722 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.171798 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.171970 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.172046 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.172209 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.172287 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.172455 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.172532 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.172702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.172939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.173204 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.173460 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.173699 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.173955 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.174204 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.174280 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.174471 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.174555 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.174730 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] auth.external = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.174807 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.174970 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.175071 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.175258 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.175345 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.175492 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] federation.caching = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.175577 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.175731 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.175809 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] auth.password = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.175980 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.176059 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] auth.token = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.176225 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.176302 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.176456 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.176533 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.176703 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.176783 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.176939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.177017 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.177193 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.177274 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.177429 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.177514 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.177675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.177751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.177912 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.177994 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] credential.caching = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.178160 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.178239 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.178395 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] identity.caching = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.178472 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.178642 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.178725 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.178890 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.178968 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.179148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.179227 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.179386 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.179464 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.179633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.179731 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.179852 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.179926 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.180094 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.180172 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.180342 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.180421 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.180594 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.180670 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.180827 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.180904 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.181059 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.181144 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.181291 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.181368 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.181534 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.181656 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.181806 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.181889 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] federation.caching = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.182018 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.182097 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.182273 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.182381 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.182569 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.182647 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.182806 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.182884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.183083 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.183178 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.183329 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.183413 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.183575 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.183652 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.183812 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.183897 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.184044 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.184122 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.184278 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.184356 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.184520 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.184605 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.184760 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.184838 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] identity.caching = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.185007 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.185085 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.185239 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.185330 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.185484 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.185560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.185723 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.185806 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.185954 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.186039 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.186201 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.186277 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.186463 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.186581 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.186659 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.186726 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.186900 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.186977 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.187154 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.187232 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.187392 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.187481 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.187631 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.187709 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.187872 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.187949 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.188114 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.188200 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.188351 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.188429 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.188601 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.188683 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.188848 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.188933 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.189080 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.189158 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.189311 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.189395 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.189567 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.189659 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.189796 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.189873 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.190027 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.190103 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.190262 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.190353 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.190495 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.190572 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.190725 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.190806 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.190965 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.191080 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.191243 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.191320 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.191473 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.191552 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.191720 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.191807 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.191965 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.192042 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.192201 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.192277 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.192440 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.192525 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.192674 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.192752 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.192906 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.192982 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.193139 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.193218 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.193379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.193456 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.193609 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.193686 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.193854 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.193932 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.194099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.194175 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.194344 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.194420 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.194589 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.194668 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.194836 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.194914 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.195092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.195179 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.195336 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.195415 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.195572 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.195650 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.195808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.195885 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.196046 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.196125 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.196282 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.196359 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.196519 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.196596 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.196760 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.196839 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.196994 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.197071 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.197239 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.197316 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.197468 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.197546 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.197714 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.197792 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.197967 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.198049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.198219 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.198299 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.198458 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.198536 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.198699 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.198815 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.198933 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.199147 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.199227 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.199384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.199466 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.199623 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.199713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.199864 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] resource.caching = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.199942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.200110 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.200186 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.200356 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.200441 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.200595 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.200679 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.200842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.200919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.201092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.201176 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.201326 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.201402 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.201571 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.201647 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.201810 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.201895 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.202048 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.202125 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.202282 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] role.caching = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.202358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.202519 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.202603 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] role.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.202749 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.202826 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.203045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.203127 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.203265 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.203353 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.203511 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.203588 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.203749 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.203825 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.203996 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.204081 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.204235 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.204312 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.204475 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.204553 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.204717 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.204865 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.204940 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.205010 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.205183 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.205260 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.205419 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.205503 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.205660 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.205738 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.205903 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.205980 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] resource.caching = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.206154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.206306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.206405 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.206546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.206648 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.206799 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.206884 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.207079 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.207161 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.207333 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.207411 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.207584 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.207673 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.207821 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.207899 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.208077 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.208155 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.208318 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] role.caching = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.208401 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.208558 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.208634 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] role.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.208792 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.208870 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.209025 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.209110 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.209256 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.209355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.209508 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.209609 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.209764 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.209849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.210013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.210091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.210249 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.210327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.210484 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.210569 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.210714 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.210792 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.210949 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] token.caching = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.211049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.211210 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.211303 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.211456 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.211534 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.211692 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.211769 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.211946 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.212031 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.212197 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.212382 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.212459 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.212624 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.212703 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.212878 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.212957 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.213125 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.213201 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.213362 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.213439 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.214128 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.214233 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.214371 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.214449 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.214618 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.214704 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.214866 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.214943 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.215121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.215200 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.215359 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.215453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.215598 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.215676 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.215835 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.215912 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.connection = **** {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.216092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.216179 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.216352 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.216442 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.216590 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.216667 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.216832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.216907 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.217066 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.217151 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.217299 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.217375 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.217534 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.217617 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] token.caching = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.217771 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.217855 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.218015 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.218091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.218252 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.218327 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.218493 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.218582 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.218727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.218805 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.218973 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.219126 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.219231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.219321 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.219474 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.219552 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.219704 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.219781 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.219941 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.220026 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.220175 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.220253 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.220346 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 15 14:28:01.220423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.220644 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.220897 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.221132 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.221380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.221683 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.221949 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.connection = **** {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.222199 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.222450 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.222694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.222940 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.223208 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.223451 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.223694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.223937 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.224187 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.224447 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.224758 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.224998 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.225251 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.225492 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.225741 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.225976 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.226222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.226460 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:01.226636 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 15 14:28:01.337838 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.338024 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.338208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.338540 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.339331 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.339563 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.347334 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.348239 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.375780 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.376010 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.376710 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.376947 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.435104 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.435335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.484778 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.484909 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.484999 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.485075 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.551588 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.551816 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.552019 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.552125 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.552254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.552336 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.552475 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.552555 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.552694 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.552785 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.553004 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.553226 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.576289 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.576517 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.576659 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.576800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.576923 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.577052 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.577175 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.577308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.577437 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.577562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.577690 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.577816 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.577942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.577942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.588791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.589677 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.601862 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.602095 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.602238 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.602367 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.602490 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.602612 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.602740 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.602872 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.603030 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.603159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.603241 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.603407 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.603529 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.603666 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:01.620613 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.620824 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 14:28:01.621163 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.621288 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.621424 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 14:28:01.621560 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 14:28:01.621683 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.621803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.621925 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 14:28:01.622053 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.622285 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.622409 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.622536 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.622662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.622819 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.623118 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.633422 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.633699 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.633966 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.634136 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.634261 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.634400 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.634670 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.634954 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.643217 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.643946 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.646612 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.646889 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.647273 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.647401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.647557 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.647692 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.647840 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.647971 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.648094 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.648247 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.648384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.648510 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.648643 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.648767 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.648889 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.649012 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.649168 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.649303 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.649456 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.649583 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.649729 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.649982 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.676528 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.677270 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.678150 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.678426 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.678983 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.679284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.683760 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.684039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.684318 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.684625 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.684906 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.685189 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.691775 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.692056 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.692713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.692997 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.696565 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.696847 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.697130 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.697410 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.697613 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.697742 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.697902 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.698514 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.698694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.698820 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.698961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.699107 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.699258 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.699538 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.699806 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.700073 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.718492 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.718774 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.719587 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.719871 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.723399 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.724485 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.727103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.727400 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.728235 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.728535 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.732586 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.733763 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.736245 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.737461 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.739979 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.740287 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.740572 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.740854 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.741130 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.741361 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.741444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.741669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.741954 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.742237 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.742516 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.742796 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.761121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.761400 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.761665 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.761924 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.762658 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.762943 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.763237 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.763503 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.770124 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.771864 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.771986 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.772155 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.773601 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.773896 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.775237 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.775525 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.777664 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.777760 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.777945 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.778042 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.778331 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.779658 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.779959 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.780247 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.791720 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.792016 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.792305 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.792583 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.793063 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.793345 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.793675 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.793799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.794079 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.794366 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.794648 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.794941 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.795248 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.795589 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.812417 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.814285 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.815714 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.817590 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.819014 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.820652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.820938 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.822449 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.822567 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.822727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.823071 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.824952 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.825227 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.825518 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.830873 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.832520 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.832920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.834636 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.835796 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.836086 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.836364 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.836633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.837930 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.838224 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.838511 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.838869 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.846075 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.846362 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.846642 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.848234 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.848524 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.848807 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.854636 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.854915 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.855219 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.855494 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.855763 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.856049 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.856322 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.856591 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.856864 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.856944 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:01.857151 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.857230 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.857434 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.857516 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.857711 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.857799 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.857984 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.858069 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.858262 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.858342 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.858548 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.858816 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.859118 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.859497 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.859771 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.860048 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.860328 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.860599 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:01.897623 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x558a01f4c9c0 pid: 80481 (default app) Jun 15 14:28:01.900150 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x558a01f4c9c0 pid: 80482 (default app) Jun 15 14:28:01.901054 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f09601-d53b-4f93-a680-44e3d45865ae None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:01.901381 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f09601-d53b-4f93-a680-44e3d45865ae None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:01.901662 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f09601-d53b-4f93-a680-44e3d45865ae None None] PATH_INFO: `/v3` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:01.905988 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1/1] 10.222.0.45 () {50 vars in 791 bytes} [Wed Jun 15 14:28:01 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:09.665022 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0237b079-b44c-4f04-b113-dca2dda4a5ec None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:09.665582 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0237b079-b44c-4f04-b113-dca2dda4a5ec None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:09.665582 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0237b079-b44c-4f04-b113-dca2dda4a5ec None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:09.670418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1/2] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:09 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:28:09.675580 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6c0e7-1d6e-49f6-ac7c-6fab8cc383b9 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:09.676009 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6c0e7-1d6e-49f6-ac7c-6fab8cc383b9 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:09.676128 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6c0e7-1d6e-49f6-ac7c-6fab8cc383b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:09.745939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: INFO dbcounter [None req-b4c6c0e7-1d6e-49f6-ac7c-6fab8cc383b9 None None] Registered counter for database keystone Jun 15 14:28:09.766873 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG oslo_db.sqlalchemy.engines [None req-b4c6c0e7-1d6e-49f6-ac7c-6fab8cc383b9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80481) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 15 14:28:09.772921 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writer thread running {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 15 14:28:09.946913 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG passlib.handlers.bcrypt [None req-b4c6c0e7-1d6e-49f6-ac7c-6fab8cc383b9 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80481) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 15 14:28:09.947282 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG passlib.handlers.bcrypt [None req-b4c6c0e7-1d6e-49f6-ac7c-6fab8cc383b9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80481) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 15 14:28:10.001545 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-b4c6c0e7-1d6e-49f6-ac7c-6fab8cc383b9 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:10.003170 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-b4c6c0e7-1d6e-49f6-ac7c-6fab8cc383b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:10.124004 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 2/3] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:09 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 449 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 15 14:28:10.304200 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0237b079-b44c-4f04-b113-dca2dda4a5ec None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:10.375465 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: INFO dbcounter [None req-0237b079-b44c-4f04-b113-dca2dda4a5ec None None] Registered counter for database keystone Jun 15 14:28:10.394941 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG oslo_db.sqlalchemy.engines [None req-0237b079-b44c-4f04-b113-dca2dda4a5ec None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80482) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 15 14:28:10.401048 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writer thread running {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 15 14:28:10.521101 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0237b079-b44c-4f04-b113-dca2dda4a5ec None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:10.525351 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c6e7ed-3bb8-4b5a-a8cd-15f06ec8887b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:10.526015 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c6e7ed-3bb8-4b5a-a8cd-15f06ec8887b admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:10.526329 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c6e7ed-3bb8-4b5a-a8cd-15f06ec8887b admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:10.526611 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c6e7ed-3bb8-4b5a-a8cd-15f06ec8887b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:10.534671 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c6e7ed-3bb8-4b5a-a8cd-15f06ec8887b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:10.587110 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: WARNING keystone.common.rbac_enforcer.enforcer [None req-04c6e7ed-3bb8-4b5a-a8cd-15f06ec8887b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 15 14:28:10.587389 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c6e7ed-3bb8-4b5a-a8cd-15f06ec8887b admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:10.597094 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 2/4] 10.222.0.45 () {58 vars in 1324 bytes} [Wed Jun 15 14:28:10 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:28:10.603133 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c6c0e7-1d6e-49f6-ac7c-6fab8cc383b9 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:10.620709 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c6c0e7-1d6e-49f6-ac7c-6fab8cc383b9 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:10.625396 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e21c673-ccf6-4d84-8029-d22d9e6bf825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:10.626481 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21c673-ccf6-4d84-8029-d22d9e6bf825 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:10.626765 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21c673-ccf6-4d84-8029-d22d9e6bf825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:10.627075 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21c673-ccf6-4d84-8029-d22d9e6bf825 admin admin] PATH_INFO: `/v3/projects/66e8228804bb44bd89c3894562c27b12` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:10.628578 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e21c673-ccf6-4d84-8029-d22d9e6bf825 admin admin] RBAC: Authorizing `identity:get_project(project_id=66e8228804bb44bd89c3894562c27b12)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:10.683697 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0e21c673-ccf6-4d84-8029-d22d9e6bf825 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 15 14:28:10.684242 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e21c673-ccf6-4d84-8029-d22d9e6bf825 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:10.688702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 3/5] 10.222.0.45 () {56 vars in 1207 bytes} [Wed Jun 15 14:28:10 2022] GET /identity/v3/projects/66e8228804bb44bd89c3894562c27b12 => generated 344 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:12.754878 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35d2d8-42a3-4c4c-b466-ce65b60faac0 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:12.755314 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35d2d8-42a3-4c4c-b466-ce65b60faac0 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:12.755489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35d2d8-42a3-4c4c-b466-ce65b60faac0 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:12.759622 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 3/6] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:12.764938 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c31c3-e4e8-4d58-a44a-1da9c53889d4 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:12.765338 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c31c3-e4e8-4d58-a44a-1da9c53889d4 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:12.765508 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c31c3-e4e8-4d58-a44a-1da9c53889d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:12.784342 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-b02c31c3-e4e8-4d58-a44a-1da9c53889d4 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:12.785861 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-b02c31c3-e4e8-4d58-a44a-1da9c53889d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:12.794410 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 4/7] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:12 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 15 14:28:12.816728 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35d2d8-42a3-4c4c-b466-ce65b60faac0 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:12.828682 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35d2d8-42a3-4c4c-b466-ce65b60faac0 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:12.832880 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff21ea-7dff-4e33-ab74-1120372c8728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:12.833508 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff21ea-7dff-4e33-ab74-1120372c8728 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:12.833797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff21ea-7dff-4e33-ab74-1120372c8728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:12.834151 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff21ea-7dff-4e33-ab74-1120372c8728 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:12.841799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ff21ea-7dff-4e33-ab74-1120372c8728 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:12.843608 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ff21ea-7dff-4e33-ab74-1120372c8728 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:12.852348 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 4/8] 10.222.0.45 () {58 vars in 1324 bytes} [Wed Jun 15 14:28:12 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:28:12.858006 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02c31c3-e4e8-4d58-a44a-1da9c53889d4 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:12.866974 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02c31c3-e4e8-4d58-a44a-1da9c53889d4 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:12.872086 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa8332-f5d6-49b4-b7b9-bd9de7368468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:12.872945 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa8332-f5d6-49b4-b7b9-bd9de7368468 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:12.873231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa8332-f5d6-49b4-b7b9-bd9de7368468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:12.873507 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa8332-f5d6-49b4-b7b9-bd9de7368468 admin admin] PATH_INFO: `/v3/users/a476d9b478cd40989e9e4e02fbc699e5` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:12.874908 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa8332-f5d6-49b4-b7b9-bd9de7368468 admin admin] RBAC: Authorizing `identity:get_user(user_id=a476d9b478cd40989e9e4e02fbc699e5)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:12.876778 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa8332-f5d6-49b4-b7b9-bd9de7368468 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:12.878355 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 5/9] 10.222.0.45 () {56 vars in 1198 bytes} [Wed Jun 15 14:28:12 2022] GET /identity/v3/users/a476d9b478cd40989e9e4e02fbc699e5 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:15.073912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df09574-2447-4223-a292-bb5dfe4cb18f None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:15.074429 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f4bb7-04e7-430a-8936-50aeeef50da5 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:15.074764 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df09574-2447-4223-a292-bb5dfe4cb18f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:15.074871 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f4bb7-04e7-430a-8936-50aeeef50da5 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:15.074957 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df09574-2447-4223-a292-bb5dfe4cb18f None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:15.075069 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f4bb7-04e7-430a-8936-50aeeef50da5 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:15.076016 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 6/11] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:28:15.076118 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 5/11] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:28:15.080632 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3e8d1-2e6e-40d8-8bae-613a1379ea54 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:15.080921 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3e8d1-2e6e-40d8-8bae-613a1379ea54 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:15.081019 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b800d-0122-4ac0-b7c2-851b957e1c9c None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:15.081210 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3e8d1-2e6e-40d8-8bae-613a1379ea54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:15.081301 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b800d-0122-4ac0-b7c2-851b957e1c9c None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:15.081527 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b800d-0122-4ac0-b7c2-851b957e1c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:15.097908 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-93a3e8d1-2e6e-40d8-8bae-613a1379ea54 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:15.099120 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-93a3e8d1-2e6e-40d8-8bae-613a1379ea54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:15.120510 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG passlib.handlers.bcrypt [None req-385b800d-0122-4ac0-b7c2-851b957e1c9c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80482) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 15 14:28:15.120937 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG passlib.handlers.bcrypt [None req-385b800d-0122-4ac0-b7c2-851b957e1c9c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80482) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 15 14:28:15.167172 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-385b800d-0122-4ac0-b7c2-851b957e1c9c None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:15.168746 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-385b800d-0122-4ac0-b7c2-851b957e1c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:15.177353 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 7/12] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:28:15.199340 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3e8d1-2e6e-40d8-8bae-613a1379ea54 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:15.222282 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:15.269166 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 6/13] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:28:15.292532 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b800d-0122-4ac0-b7c2-851b957e1c9c None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:15.303346 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3e8d1-2e6e-40d8-8bae-613a1379ea54 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:15.342410 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0522e887-68cd-41e0-a1a2-14f98d07c52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:15.343120 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522e887-68cd-41e0-a1a2-14f98d07c52e None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:15.343493 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522e887-68cd-41e0-a1a2-14f98d07c52e None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:15.343839 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522e887-68cd-41e0-a1a2-14f98d07c52e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:15.403463 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b800d-0122-4ac0-b7c2-851b957e1c9c None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:15.405734 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:15.415161 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0522e887-68cd-41e0-a1a2-14f98d07c52e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:15.416661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0522e887-68cd-41e0-a1a2-14f98d07c52e None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:15.439698 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:15.443963 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05093138-205f-4903-829b-4c5c37c5dbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:15.444992 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05093138-205f-4903-829b-4c5c37c5dbe1 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:15.445275 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05093138-205f-4903-829b-4c5c37c5dbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:15.445546 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05093138-205f-4903-829b-4c5c37c5dbe1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:15.457772 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05093138-205f-4903-829b-4c5c37c5dbe1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:15.468989 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-05093138-205f-4903-829b-4c5c37c5dbe1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:28:15.468989 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:15.468989 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:15.468989 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:15.468989 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:15.468989 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:15.468989 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:15.468989 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:15.468989 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:15.468989 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:15.468989 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:15.468989 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:15.468989 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:15.468989 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:15.468989 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:15.468989 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:15.470049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:15.470049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:15.470049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:15.470049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:15.470049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:15.470049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:15.470049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:15.470049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:15.470049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:15.470049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:15.470049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:15.470049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:15.470049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:15.470049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:15.470049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:15.470049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:15.471129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:15.471129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:15.471129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:15.471129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:15.471129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:15.471129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:15.471129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:15.471129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:15.471129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:15.471129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:15.471129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:28:15.471129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:28:15.471129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 7/14] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:28:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:28:15.476180 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05093138-205f-4903-829b-4c5c37c5dbe1 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:15.528453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 8/15] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:28:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 14:28:15.533894 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0522e887-68cd-41e0-a1a2-14f98d07c52e None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:15.580818 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05093138-205f-4903-829b-4c5c37c5dbe1 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:15.613096 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c9f63-da22-4c58-9da2-2ee8308907a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:15.613684 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c9f63-da22-4c58-9da2-2ee8308907a6 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:15.613968 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c9f63-da22-4c58-9da2-2ee8308907a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:15.614245 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c9f63-da22-4c58-9da2-2ee8308907a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:15.615008 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c9f63-da22-4c58-9da2-2ee8308907a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:15.615324 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c9f63-da22-4c58-9da2-2ee8308907a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:15.616765 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c9f63-da22-4c58-9da2-2ee8308907a6 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:15.629943 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0522e887-68cd-41e0-a1a2-14f98d07c52e None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:15.630162 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 8/16] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:28:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:15.636354 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c9f63-da22-4c58-9da2-2ee8308907a6 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:15.653188 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:15.664923 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a47f69-78b3-444d-859d-4144b244b017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:15.665608 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:15.665740 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a47f69-78b3-444d-859d-4144b244b017 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:15.665983 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a47f69-78b3-444d-859d-4144b244b017 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:15.666496 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a47f69-78b3-444d-859d-4144b244b017 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:15.677087 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a47f69-78b3-444d-859d-4144b244b017 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:15.678591 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a47f69-78b3-444d-859d-4144b244b017 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:15.703546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-43a47f69-78b3-444d-859d-4144b244b017 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:28:15.703546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:15.703546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:15.703546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:15.703546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:15.703546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:15.703546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:15.703546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:15.703546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:15.703546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:15.703546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:15.703546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:15.703546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:28:15.703546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:28:15.703546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:28:15.704379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:28:15.704379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:15.704379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:15.704379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:15.704379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:15.704379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:15.704379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:15.704379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:15.704379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:15.704379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:15.704379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:15.704379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:15.704379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:15.704379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:15.704379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:15.704379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:15.704379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:15.705148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:28:15.705148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:28:15.705148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:28:15.705148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:28:15.705148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 9/17] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:28:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:28:15.710355 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a47f69-78b3-444d-859d-4144b244b017 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:15.749445 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c9f63-da22-4c58-9da2-2ee8308907a6 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:15.782502 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb80e0-a3ba-4eba-a17c-10dc46e5d355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:15.783405 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb80e0-a3ba-4eba-a17c-10dc46e5d355 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:15.783691 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb80e0-a3ba-4eba-a17c-10dc46e5d355 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:15.783969 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb80e0-a3ba-4eba-a17c-10dc46e5d355 None admin] PATH_INFO: `/v3/users/a476d9b478cd40989e9e4e02fbc699e5` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:15.785504 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb80e0-a3ba-4eba-a17c-10dc46e5d355 None admin] RBAC: Authorizing `identity:get_user(user_id=a476d9b478cd40989e9e4e02fbc699e5)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:15.786959 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb80e0-a3ba-4eba-a17c-10dc46e5d355 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:15.788612 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 9/18] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:15 2022] GET /identity/v3/users/a476d9b478cd40989e9e4e02fbc699e5 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:15.794490 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb80e0-a3ba-4eba-a17c-10dc46e5d355 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:15.813062 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a47f69-78b3-444d-859d-4144b244b017 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:15.846352 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512369e3-a59a-4cf4-92d4-149075415b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:15.847036 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512369e3-a59a-4cf4-92d4-149075415b9d None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:15.847326 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512369e3-a59a-4cf4-92d4-149075415b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:15.847604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512369e3-a59a-4cf4-92d4-149075415b9d None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:15.848331 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512369e3-a59a-4cf4-92d4-149075415b9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:15.848642 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512369e3-a59a-4cf4-92d4-149075415b9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:15.850031 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512369e3-a59a-4cf4-92d4-149075415b9d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:15.867319 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 10/19] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:28:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:15.887021 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:15.895016 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb80e0-a3ba-4eba-a17c-10dc46e5d355 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:15.931216 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3cc68-5e93-4862-965e-c3e47ff0d302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:15.932027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3cc68-5e93-4862-965e-c3e47ff0d302 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:15.932310 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3cc68-5e93-4862-965e-c3e47ff0d302 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:15.933248 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3cc68-5e93-4862-965e-c3e47ff0d302 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:15.934500 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d3cc68-5e93-4862-965e-c3e47ff0d302 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:15.936021 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d3cc68-5e93-4862-965e-c3e47ff0d302 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:15.937729 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 10/20] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:28:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:15.943846 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512369e3-a59a-4cf4-92d4-149075415b9d None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:15.978921 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:16.049536 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512369e3-a59a-4cf4-92d4-149075415b9d None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:16.083084 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d418a6ac-c053-475c-adc0-08807f8f97ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:16.083754 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418a6ac-c053-475c-adc0-08807f8f97ca None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:16.084041 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418a6ac-c053-475c-adc0-08807f8f97ca None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:16.084320 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418a6ac-c053-475c-adc0-08807f8f97ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:16.085158 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d418a6ac-c053-475c-adc0-08807f8f97ca None admin] RBAC: Adding query filter params (role.id=ed1cfa2fcf724a4d84f5acea100930c7, scope.domain.id=default, user.id=a476d9b478cd40989e9e4e02fbc699e5) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:16.085476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d418a6ac-c053-475c-adc0-08807f8f97ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:16.086952 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d418a6ac-c053-475c-adc0-08807f8f97ca None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:16.094844 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 11/21] 10.222.0.45 () {56 vars in 1322 bytes} [Wed Jun 15 14:28:15 2022] GET /identity/v3/role_assignments?user.id=a476d9b478cd40989e9e4e02fbc699e5&scope.domain.id=default&role.id=ed1cfa2fcf724a4d84f5acea100930c7 => generated 233 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:18.402870 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de0081-b403-4565-974e-61f92c8f7170 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:18.403667 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de0081-b403-4565-974e-61f92c8f7170 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:18.403667 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de0081-b403-4565-974e-61f92c8f7170 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:18.405079 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 11/22] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:28:18.413954 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa0ef0-c5bb-49f5-932f-62a54e063d3f None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:18.414836 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa0ef0-c5bb-49f5-932f-62a54e063d3f None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:18.414836 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa0ef0-c5bb-49f5-932f-62a54e063d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:18.456057 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-adfa0ef0-c5bb-49f5-932f-62a54e063d3f None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:18.457286 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-adfa0ef0-c5bb-49f5-932f-62a54e063d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:18.541224 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 12/23] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:28:18.560019 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf39d6-0012-4620-9d4e-78aa5250e371 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:18.560311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf39d6-0012-4620-9d4e-78aa5250e371 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:18.560596 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf39d6-0012-4620-9d4e-78aa5250e371 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:18.561994 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 12/24] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:18.567893 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af6fb2-7049-4b48-966a-12a5fdd68723 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:18.568184 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af6fb2-7049-4b48-966a-12a5fdd68723 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:18.568467 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af6fb2-7049-4b48-966a-12a5fdd68723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:18.580359 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf39d6-0012-4620-9d4e-78aa5250e371 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:18.595068 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-90af6fb2-7049-4b48-966a-12a5fdd68723 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:18.596266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-90af6fb2-7049-4b48-966a-12a5fdd68723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:18.636422 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:18.770739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 13/25] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:28:18.772123 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c584e2e-d072-4afd-b1e0-411cf9565a93 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:18.772417 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c584e2e-d072-4afd-b1e0-411cf9565a93 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:18.772694 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c584e2e-d072-4afd-b1e0-411cf9565a93 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:18.774086 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 14/26] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:18.779885 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97fd84-324b-48fa-8ac1-e9ff72678e00 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:18.780170 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97fd84-324b-48fa-8ac1-e9ff72678e00 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:18.780446 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97fd84-324b-48fa-8ac1-e9ff72678e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:18.819942 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-cb97fd84-324b-48fa-8ac1-e9ff72678e00 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:18.821133 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-cb97fd84-324b-48fa-8ac1-e9ff72678e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:18.851448 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf39d6-0012-4620-9d4e-78aa5250e371 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:18.896714 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 15/27] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:28:18.897547 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97fd84-324b-48fa-8ac1-e9ff72678e00 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:18.924631 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d600bf-51ef-41f4-bbf1-5689077e4e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:18.925442 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d600bf-51ef-41f4-bbf1-5689077e4e45 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:18.925731 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d600bf-51ef-41f4-bbf1-5689077e4e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:18.926009 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d600bf-51ef-41f4-bbf1-5689077e4e45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:18.966024 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d600bf-51ef-41f4-bbf1-5689077e4e45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:18.967568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d600bf-51ef-41f4-bbf1-5689077e4e45 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:18.989171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:19.003669 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97fd84-324b-48fa-8ac1-e9ff72678e00 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:19.004488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-f6d600bf-51ef-41f4-bbf1-5689077e4e45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:28:19.004488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:19.004488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:19.004488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:19.004488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:19.004488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:19.004488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:19.004488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:19.004488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:19.004488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:19.004488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:19.004488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:19.004488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:28:19.004488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:28:19.004488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:28:19.005270 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:28:19.005270 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:19.005270 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:19.005270 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:19.005270 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:19.005270 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:19.005270 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:19.005270 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:19.005270 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:19.005270 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:19.005270 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:19.005270 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:19.005270 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:19.005270 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:19.005270 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:19.005270 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:19.005270 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:19.006062 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:28:19.006062 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:28:19.006062 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:28:19.006062 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:28:19.006062 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 13/28] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:28:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 426 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:28:19.007054 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d600bf-51ef-41f4-bbf1-5689077e4e45 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:19.040956 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f09a7-3cd6-4ead-9f01-f67b7a1014f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:19.041631 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f09a7-3cd6-4ead-9f01-f67b7a1014f4 None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:19.042524 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f09a7-3cd6-4ead-9f01-f67b7a1014f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:19.042803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f09a7-3cd6-4ead-9f01-f67b7a1014f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:19.043591 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0f09a7-3cd6-4ead-9f01-f67b7a1014f4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:19.045104 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0f09a7-3cd6-4ead-9f01-f67b7a1014f4 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:19.048074 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:19.265517 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d600bf-51ef-41f4-bbf1-5689077e4e45 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:19.296740 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 16/29] 10.222.0.45 () {60 vars in 1131 bytes} [Wed Jun 15 14:28:18 2022] POST /identity/v3/roles => generated 219 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:28:19.297729 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f09a7-3cd6-4ead-9f01-f67b7a1014f4 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:19.355749 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a89493-4b61-4b43-acfd-1e9f3ea02885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:19.356848 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a89493-4b61-4b43-acfd-1e9f3ea02885 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:19.357274 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a89493-4b61-4b43-acfd-1e9f3ea02885 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:19.357768 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a89493-4b61-4b43-acfd-1e9f3ea02885 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:19.382028 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a89493-4b61-4b43-acfd-1e9f3ea02885 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:19.390929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-d1a89493-4b61-4b43-acfd-1e9f3ea02885 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:28:19.390929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:19.390929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:19.390929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:19.390929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:19.390929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:19.390929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:19.390929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:19.390929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:19.390929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:19.390929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:19.390929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:19.390929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:19.390929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:19.390929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:19.390929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:19.392363 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:19.392363 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:19.392363 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:19.392363 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:19.392363 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:19.392363 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:19.392363 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:19.392363 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:19.392363 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:19.392363 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:19.392363 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:19.392363 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:19.392363 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:19.392363 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:19.392363 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:19.392363 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:19.393307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:19.393307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:19.393307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:19.393307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:19.393307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:19.393307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:19.393307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:19.393307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:19.393307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:19.393307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:19.393307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:28:19.393307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:28:19.400728 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 14/30] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:28:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:28:19.402443 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af5d2c-d76b-45ab-a417-8674a0f7462d None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:19.402868 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af5d2c-d76b-45ab-a417-8674a0f7462d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:19.403313 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af5d2c-d76b-45ab-a417-8674a0f7462d None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:19.405221 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 15/31] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:19.412702 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af5d2c-d76b-45ab-a417-8674a0f7462d None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:19.482214 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f09a7-3cd6-4ead-9f01-f67b7a1014f4 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:19.544364 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddfc847-f020-4afc-9451-6316a5997d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:19.545065 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfc847-f020-4afc-9451-6316a5997d40 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:19.545353 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfc847-f020-4afc-9451-6316a5997d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:19.545694 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfc847-f020-4afc-9451-6316a5997d40 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:19.546451 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddfc847-f020-4afc-9451-6316a5997d40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:19.546765 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddfc847-f020-4afc-9451-6316a5997d40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:19.548335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddfc847-f020-4afc-9451-6316a5997d40 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:19.561455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:19.565795 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 17/32] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:28:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:19.568432 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b6823-4495-47d8-a391-c78034e9c6f2 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:19.568642 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b6823-4495-47d8-a391-c78034e9c6f2 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:19.568998 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b6823-4495-47d8-a391-c78034e9c6f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:19.588937 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-d01b6823-4495-47d8-a391-c78034e9c6f2 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:19.590503 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d01b6823-4495-47d8-a391-c78034e9c6f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:19.594428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:19.599681 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af5d2c-d76b-45ab-a417-8674a0f7462d None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:19.650048 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9cdb34-8e24-4dd0-ab6a-187e2dd88b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:19.650985 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cdb34-8e24-4dd0-ab6a-187e2dd88b25 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:19.651465 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cdb34-8e24-4dd0-ab6a-187e2dd88b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:19.651914 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cdb34-8e24-4dd0-ab6a-187e2dd88b25 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:19.652813 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9cdb34-8e24-4dd0-ab6a-187e2dd88b25 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:19.653444 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9cdb34-8e24-4dd0-ab6a-187e2dd88b25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:19.654977 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9cdb34-8e24-4dd0-ab6a-187e2dd88b25 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:19.665527 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 16/33] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:28:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:19.666688 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9cdb34-8e24-4dd0-ab6a-187e2dd88b25 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:19.686159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 18/34] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:19.687169 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b6823-4495-47d8-a391-c78034e9c6f2 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:19.812509 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9cdb34-8e24-4dd0-ab6a-187e2dd88b25 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:19.813255 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b6823-4495-47d8-a391-c78034e9c6f2 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:19.852065 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b36ac3-abf4-4f60-9d3b-b445af3442a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:19.852963 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b36ac3-abf4-4f60-9d3b-b445af3442a9 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:19.853257 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b36ac3-abf4-4f60-9d3b-b445af3442a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:19.853534 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b36ac3-abf4-4f60-9d3b-b445af3442a9 None admin] PATH_INFO: `/v3/users/a476d9b478cd40989e9e4e02fbc699e5` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:19.854929 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b36ac3-abf4-4f60-9d3b-b445af3442a9 None admin] RBAC: Authorizing `identity:get_user(user_id=a476d9b478cd40989e9e4e02fbc699e5)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:19.856453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b36ac3-abf4-4f60-9d3b-b445af3442a9 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:19.857850 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77abdd16-f645-489f-abae-16aa99022e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:19.858427 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 19/35] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:19 2022] GET /identity/v3/users/a476d9b478cd40989e9e4e02fbc699e5 => generated 247 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:19.858658 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abdd16-f645-489f-abae-16aa99022e61 None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:19.858949 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abdd16-f645-489f-abae-16aa99022e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:19.859323 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abdd16-f645-489f-abae-16aa99022e61 None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:19.859431 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b36ac3-abf4-4f60-9d3b-b445af3442a9 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:19.860046 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77abdd16-f645-489f-abae-16aa99022e61 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:19.861496 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77abdd16-f645-489f-abae-16aa99022e61 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:19.868564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:19.869188 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:19.965957 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b36ac3-abf4-4f60-9d3b-b445af3442a9 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:19.998928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c755d-7336-4913-bd25-54b80347dcb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:19.999637 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c755d-7336-4913-bd25-54b80347dcb3 None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:19.999924 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c755d-7336-4913-bd25-54b80347dcb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:20.000204 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c755d-7336-4913-bd25-54b80347dcb3 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:20.000953 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1c755d-7336-4913-bd25-54b80347dcb3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:20.002347 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1c755d-7336-4913-bd25-54b80347dcb3 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:20.011528 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 17/36] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Jun 15 14:28:19 2022] POST /identity/v3/projects => generated 301 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:28:20.013309 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc1aa5-b045-4013-ae4c-2ffc5bd60f3d None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:20.013652 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc1aa5-b045-4013-ae4c-2ffc5bd60f3d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:20.013970 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc1aa5-b045-4013-ae4c-2ffc5bd60f3d None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:20.016045 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 18/37] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:20.016984 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc1aa5-b045-4013-ae4c-2ffc5bd60f3d None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:20.026979 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 20/38] 10.222.0.45 () {60 vars in 1131 bytes} [Wed Jun 15 14:28:19 2022] POST /identity/v3/roles => generated 225 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:28:20.040737 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a5331-a00a-4d58-8cf7-ab8f19f97dfb None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:20.041054 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a5331-a00a-4d58-8cf7-ab8f19f97dfb None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:20.041316 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a5331-a00a-4d58-8cf7-ab8f19f97dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:20.080351 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-0e0a5331-a00a-4d58-8cf7-ab8f19f97dfb None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:20.081496 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-0e0a5331-a00a-4d58-8cf7-ab8f19f97dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:20.169355 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:20.172567 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc1aa5-b045-4013-ae4c-2ffc5bd60f3d None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:20.175404 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 21/39] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:20.176780 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42b456-27cc-4d57-bfbf-7522907e499e None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:20.177076 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42b456-27cc-4d57-bfbf-7522907e499e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:20.177362 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42b456-27cc-4d57-bfbf-7522907e499e None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:20.178859 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 22/40] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:20.187037 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4977a3f-ddcf-4194-a816-449dd436fed9 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:20.187371 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4977a3f-ddcf-4194-a816-449dd436fed9 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:20.187621 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4977a3f-ddcf-4194-a816-449dd436fed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:20.220702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-e4977a3f-ddcf-4194-a816-449dd436fed9 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:20.221191 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a2216-c3d9-40ab-92dc-9ae95572784d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:20.221931 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e4977a3f-ddcf-4194-a816-449dd436fed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:20.222052 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a2216-c3d9-40ab-92dc-9ae95572784d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:20.222280 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a2216-c3d9-40ab-92dc-9ae95572784d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:20.222556 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a2216-c3d9-40ab-92dc-9ae95572784d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:20.223987 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491a2216-c3d9-40ab-92dc-9ae95572784d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:20.225595 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491a2216-c3d9-40ab-92dc-9ae95572784d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:20.227670 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 19/41] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:28:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:20.228493 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a2216-c3d9-40ab-92dc-9ae95572784d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:20.304064 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:20.308359 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 23/42] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:20.309289 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4977a3f-ddcf-4194-a816-449dd436fed9 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:20.353800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a2216-c3d9-40ab-92dc-9ae95572784d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:20.411584 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4977a3f-ddcf-4194-a816-449dd436fed9 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:20.440704 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:20.450566 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34a3bd-e0fc-4f65-bd26-3640c95657ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:20.451490 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34a3bd-e0fc-4f65-bd26-3640c95657ef None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:20.451777 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34a3bd-e0fc-4f65-bd26-3640c95657ef None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:20.452055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34a3bd-e0fc-4f65-bd26-3640c95657ef None admin] PATH_INFO: `/v3/domains/default/users/a476d9b478cd40989e9e4e02fbc699e5/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:20.454111 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf34a3bd-e0fc-4f65-bd26-3640c95657ef None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=a476d9b478cd40989e9e4e02fbc699e5, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:20.455656 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf34a3bd-e0fc-4f65-bd26-3640c95657ef None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:20.483390 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 24/43] 10.222.0.45 () {58 vars in 1361 bytes} [Wed Jun 15 14:28:20 2022] PUT /identity/v3/domains/default/users/a476d9b478cd40989e9e4e02fbc699e5/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:28:20.483390 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34a3bd-e0fc-4f65-bd26-3640c95657ef None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:20.499320 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9911f6-8db1-46be-94a0-91bbf97aed40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:20.500163 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9911f6-8db1-46be-94a0-91bbf97aed40 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:20.500451 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9911f6-8db1-46be-94a0-91bbf97aed40 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:20.500730 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9911f6-8db1-46be-94a0-91bbf97aed40 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:20.501990 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9911f6-8db1-46be-94a0-91bbf97aed40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:20.503530 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9911f6-8db1-46be-94a0-91bbf97aed40 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:20.505441 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 20/44] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:28:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:20.511038 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9911f6-8db1-46be-94a0-91bbf97aed40 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:20.611413 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34a3bd-e0fc-4f65-bd26-3640c95657ef None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:20.644408 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9911f6-8db1-46be-94a0-91bbf97aed40 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:20.651078 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d07e6-52fd-457a-be4c-71f6c8f76702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:20.652300 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d07e6-52fd-457a-be4c-71f6c8f76702 None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:20.652590 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d07e6-52fd-457a-be4c-71f6c8f76702 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:20.652869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d07e6-52fd-457a-be4c-71f6c8f76702 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:20.653645 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45d07e6-52fd-457a-be4c-71f6c8f76702 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:20.655889 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45d07e6-52fd-457a-be4c-71f6c8f76702 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:20.678650 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 25/45] 10.222.0.45 () {60 vars in 1131 bytes} [Wed Jun 15 14:28:20 2022] POST /identity/v3/roles => generated 223 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:28:20.679415 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:20.691213 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c1e3b-005b-4328-b36a-2c50ea9ab746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:20.691916 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c1e3b-005b-4328-b36a-2c50ea9ab746 None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:20.692210 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c1e3b-005b-4328-b36a-2c50ea9ab746 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:20.692497 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c1e3b-005b-4328-b36a-2c50ea9ab746 None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:20.693247 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428c1e3b-005b-4328-b36a-2c50ea9ab746 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:20.694933 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428c1e3b-005b-4328-b36a-2c50ea9ab746 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:20.730013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 21/46] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Jun 15 14:28:20 2022] POST /identity/v3/projects => generated 312 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:28:22.648136 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9381bb1-9f46-4acd-b1eb-c997a92a7737 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:22.648675 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9381bb1-9f46-4acd-b1eb-c997a92a7737 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:22.648675 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9381bb1-9f46-4acd-b1eb-c997a92a7737 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:22.650080 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 26/47] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:22.655168 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728aa7bb-d540-4445-bdb0-676e459f8481 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:22.655575 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728aa7bb-d540-4445-bdb0-676e459f8481 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:22.655747 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728aa7bb-d540-4445-bdb0-676e459f8481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:22.672935 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-728aa7bb-d540-4445-bdb0-676e459f8481 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:22.674511 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-728aa7bb-d540-4445-bdb0-676e459f8481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:22.743950 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 22/48] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:22.768025 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9381bb1-9f46-4acd-b1eb-c997a92a7737 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:22.815546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:22.876740 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9381bb1-9f46-4acd-b1eb-c997a92a7737 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:22.889777 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcd488-30c3-4bc3-9b9b-21c706a9db83 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:22.890072 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcd488-30c3-4bc3-9b9b-21c706a9db83 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:22.890354 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcd488-30c3-4bc3-9b9b-21c706a9db83 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:22.892026 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 23/49] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:28:22.896845 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b36be-ffd5-4439-9ead-b69d07770298 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:22.897142 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b36be-ffd5-4439-9ead-b69d07770298 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:22.897435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b36be-ffd5-4439-9ead-b69d07770298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:22.911048 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6ed6a-fe92-4ffb-8d2a-6b616438b342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:22.911778 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6ed6a-fe92-4ffb-8d2a-6b616438b342 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:22.912063 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6ed6a-fe92-4ffb-8d2a-6b616438b342 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:22.912344 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6ed6a-fe92-4ffb-8d2a-6b616438b342 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:22.914005 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-083b36be-ffd5-4439-9ead-b69d07770298 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:22.915222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-083b36be-ffd5-4439-9ead-b69d07770298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:22.924332 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba6ed6a-fe92-4ffb-8d2a-6b616438b342 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:22.933617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-aba6ed6a-fe92-4ffb-8d2a-6b616438b342 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:28:22.933617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:22.933617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:22.933617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:22.933617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:22.933617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:22.933617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:22.933617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:22.933617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:22.933617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:22.933617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:22.933617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:22.933617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:22.933617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:22.933617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:22.933617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:22.934610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:22.934610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:22.934610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:22.934610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:22.934610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:22.934610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:22.934610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:22.934610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:22.934610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:22.934610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:22.934610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:22.934610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:22.934610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:22.934610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:22.934610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:22.934610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:22.935372 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:22.935372 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:22.935372 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:22.935372 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:22.935372 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:22.935372 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:22.935372 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:22.935372 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:22.935372 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:22.935372 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:22.935372 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:28:22.935372 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:28:22.935372 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 27/50] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:28:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:28:22.940557 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6ed6a-fe92-4ffb-8d2a-6b616438b342 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:22.982222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 24/51] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:28:23.005532 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b36be-ffd5-4439-9ead-b69d07770298 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:23.038353 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6ed6a-fe92-4ffb-8d2a-6b616438b342 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:23.040819 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:23.045348 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:23.072506 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da503c1c-bf09-408c-98ec-1ea9eae538a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:23.073936 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da503c1c-bf09-408c-98ec-1ea9eae538a6 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:23.074525 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da503c1c-bf09-408c-98ec-1ea9eae538a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:23.074525 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da503c1c-bf09-408c-98ec-1ea9eae538a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:23.075318 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da503c1c-bf09-408c-98ec-1ea9eae538a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:23.075635 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da503c1c-bf09-408c-98ec-1ea9eae538a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:23.077065 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da503c1c-bf09-408c-98ec-1ea9eae538a6 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:23.088676 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 28/52] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:28:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:23.094074 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da503c1c-bf09-408c-98ec-1ea9eae538a6 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:23.106173 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b36be-ffd5-4439-9ead-b69d07770298 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:23.141639 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f672a0a-a069-4108-aa0d-e348c7e67848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:23.142436 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f672a0a-a069-4108-aa0d-e348c7e67848 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:23.142722 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f672a0a-a069-4108-aa0d-e348c7e67848 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:23.143022 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f672a0a-a069-4108-aa0d-e348c7e67848 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:23.144212 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f672a0a-a069-4108-aa0d-e348c7e67848 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:23.145686 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f672a0a-a069-4108-aa0d-e348c7e67848 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:23.147356 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 25/53] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:28:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:23.152929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f672a0a-a069-4108-aa0d-e348c7e67848 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:23.192380 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da503c1c-bf09-408c-98ec-1ea9eae538a6 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:23.224794 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13442e0a-6732-478d-a7a0-f35628f28c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:23.225665 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13442e0a-6732-478d-a7a0-f35628f28c28 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:23.225948 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13442e0a-6732-478d-a7a0-f35628f28c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:23.226221 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13442e0a-6732-478d-a7a0-f35628f28c28 None admin] PATH_INFO: `/v3/users/a476d9b478cd40989e9e4e02fbc699e5` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:23.227456 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13442e0a-6732-478d-a7a0-f35628f28c28 None admin] RBAC: Authorizing `identity:get_user(user_id=a476d9b478cd40989e9e4e02fbc699e5)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:23.228931 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13442e0a-6732-478d-a7a0-f35628f28c28 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:23.231122 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 29/54] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:23 2022] GET /identity/v3/users/a476d9b478cd40989e9e4e02fbc699e5 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:23.236300 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13442e0a-6732-478d-a7a0-f35628f28c28 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:23.254011 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f672a0a-a069-4108-aa0d-e348c7e67848 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:23.256319 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:23.270600 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:23.288329 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3916238c-1634-4172-a3f3-ffae86702b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:23.288991 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916238c-1634-4172-a3f3-ffae86702b25 None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:23.289279 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916238c-1634-4172-a3f3-ffae86702b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:23.289561 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916238c-1634-4172-a3f3-ffae86702b25 None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:23.290267 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3916238c-1634-4172-a3f3-ffae86702b25 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:23.291757 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3916238c-1634-4172-a3f3-ffae86702b25 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:23.316201 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 26/55] 10.222.0.45 () {60 vars in 1140 bytes} [Wed Jun 15 14:28:23 2022] POST /identity/v3/projects => generated 298 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:28:23.337979 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13442e0a-6732-478d-a7a0-f35628f28c28 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:23.375285 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f3978a-e4c5-47a4-862d-99cf6ea1180f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:23.376105 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3978a-e4c5-47a4-862d-99cf6ea1180f None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:23.376391 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3978a-e4c5-47a4-862d-99cf6ea1180f None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:23.376669 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3978a-e4c5-47a4-862d-99cf6ea1180f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:23.377906 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f3978a-e4c5-47a4-862d-99cf6ea1180f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:23.379428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f3978a-e4c5-47a4-862d-99cf6ea1180f None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:23.381101 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 30/56] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:28:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:23.387451 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3916238c-1634-4172-a3f3-ffae86702b25 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:23.493704 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3916238c-1634-4172-a3f3-ffae86702b25 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:23.527493 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807d2d1-d6da-40b4-ac3f-5cffff1d8060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:23.528156 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807d2d1-d6da-40b4-ac3f-5cffff1d8060 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:23.528441 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807d2d1-d6da-40b4-ac3f-5cffff1d8060 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:23.528717 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807d2d1-d6da-40b4-ac3f-5cffff1d8060 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:23.529551 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807d2d1-d6da-40b4-ac3f-5cffff1d8060 None admin] RBAC: Adding query filter params (role.id=ed1cfa2fcf724a4d84f5acea100930c7, scope.domain.id=default, user.id=a476d9b478cd40989e9e4e02fbc699e5) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:23.529862 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807d2d1-d6da-40b4-ac3f-5cffff1d8060 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:23.531348 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807d2d1-d6da-40b4-ac3f-5cffff1d8060 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:23.540319 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 27/57] 10.222.0.45 () {56 vars in 1322 bytes} [Wed Jun 15 14:28:23 2022] GET /identity/v3/role_assignments?user.id=a476d9b478cd40989e9e4e02fbc699e5&scope.domain.id=default&role.id=ed1cfa2fcf724a4d84f5acea100930c7 => generated 530 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:25.457861 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88956100-ee52-4242-8fa7-58dbc4040cf1 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:25.458364 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88956100-ee52-4242-8fa7-58dbc4040cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:25.458475 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88956100-ee52-4242-8fa7-58dbc4040cf1 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:25.459849 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 31/58] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:25.465651 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc3b83-398a-4042-86e4-cae24e8d8829 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:25.466052 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc3b83-398a-4042-86e4-cae24e8d8829 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:25.466214 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc3b83-398a-4042-86e4-cae24e8d8829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:25.482384 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-c6dc3b83-398a-4042-86e4-cae24e8d8829 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:25.483582 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c6dc3b83-398a-4042-86e4-cae24e8d8829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:25.497252 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:25.549165 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 28/59] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:25.572927 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88956100-ee52-4242-8fa7-58dbc4040cf1 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:25.675716 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88956100-ee52-4242-8fa7-58dbc4040cf1 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:25.678005 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:25.708776 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65838ca-1e81-4f2b-b43d-e98765e46e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:25.709508 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65838ca-1e81-4f2b-b43d-e98765e46e7f None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:25.709797 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65838ca-1e81-4f2b-b43d-e98765e46e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:25.710078 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65838ca-1e81-4f2b-b43d-e98765e46e7f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:25.711298 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65838ca-1e81-4f2b-b43d-e98765e46e7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:25.712765 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65838ca-1e81-4f2b-b43d-e98765e46e7f None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:25.714956 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 32/60] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:28:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:25.720522 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc3b83-398a-4042-86e4-cae24e8d8829 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:25.815639 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc3b83-398a-4042-86e4-cae24e8d8829 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:25.847807 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629e2c63-13be-436f-9e78-c63b1db562d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:25.848466 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e2c63-13be-436f-9e78-c63b1db562d9 None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:25.848854 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e2c63-13be-436f-9e78-c63b1db562d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:25.849141 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e2c63-13be-436f-9e78-c63b1db562d9 None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:25.849856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629e2c63-13be-436f-9e78-c63b1db562d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:25.851331 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629e2c63-13be-436f-9e78-c63b1db562d9 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:25.875582 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 29/61] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Jun 15 14:28:25 2022] POST /identity/v3/users => generated 275 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:28:29.181919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcb47f-c481-4a19-a116-e53ab416d14e None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:29.182966 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcb47f-c481-4a19-a116-e53ab416d14e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:29.183476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcb47f-c481-4a19-a116-e53ab416d14e None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:29.185098 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 33/62] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:29.190456 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84212c4b-ec9a-48f6-be5c-330cd8b23e5e None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:29.190866 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84212c4b-ec9a-48f6-be5c-330cd8b23e5e None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:29.191054 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84212c4b-ec9a-48f6-be5c-330cd8b23e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:29.211036 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:29.220895 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-84212c4b-ec9a-48f6-be5c-330cd8b23e5e None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:29.222146 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-84212c4b-ec9a-48f6-be5c-330cd8b23e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:29.297661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3359a2-bad2-4e3e-a287-142461d389ad None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:29.297946 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3359a2-bad2-4e3e-a287-142461d389ad None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:29.298230 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3359a2-bad2-4e3e-a287-142461d389ad None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:29.299681 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 34/63] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:29.305509 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b279e-8754-4161-a8ca-4c55cba0e48c None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:29.306148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b279e-8754-4161-a8ca-4c55cba0e48c None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:29.306646 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b279e-8754-4161-a8ca-4c55cba0e48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:29.338011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-f95b279e-8754-4161-a8ca-4c55cba0e48c None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:29.340253 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 30/64] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:29.342042 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f95b279e-8754-4161-a8ca-4c55cba0e48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:29.375116 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84212c4b-ec9a-48f6-be5c-330cd8b23e5e None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:29.425778 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 35/65] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:28:29.427452 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038594c-d05b-4c3c-adea-c60b1e65ca79 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:29.427931 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038594c-d05b-4c3c-adea-c60b1e65ca79 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:29.428414 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038594c-d05b-4c3c-adea-c60b1e65ca79 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:29.429870 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 36/66] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:29.438515 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f5b00-c458-4e70-826c-05929bbd73b0 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:29.439060 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f5b00-c458-4e70-826c-05929bbd73b0 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:29.439530 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f5b00-c458-4e70-826c-05929bbd73b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:29.466351 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-ba7f5b00-c458-4e70-826c-05929bbd73b0 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:29.467871 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ba7f5b00-c458-4e70-826c-05929bbd73b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:29.488756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84212c4b-ec9a-48f6-be5c-330cd8b23e5e None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:29.534559 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec68c8-86b8-414f-8bec-60e17d6befdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:29.535419 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec68c8-86b8-414f-8bec-60e17d6befdb None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:29.535712 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec68c8-86b8-414f-8bec-60e17d6befdb None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:29.535998 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec68c8-86b8-414f-8bec-60e17d6befdb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:29.547434 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:29.549232 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec68c8-86b8-414f-8bec-60e17d6befdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:29.551193 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 37/67] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:29.552329 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f5b00-c458-4e70-826c-05929bbd73b0 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:29.565154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-ebec68c8-86b8-414f-8bec-60e17d6befdb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:28:29.565154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:29.565154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:29.565154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:29.565154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:29.565154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:29.565154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:29.565154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:29.565154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:29.565154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:29.565154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:29.565154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:29.565154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:29.565154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:29.565154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:29.565154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:29.566021 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:29.566021 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:29.566021 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:29.566021 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:29.566021 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:29.566021 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:29.566021 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:29.566021 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:29.566021 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:29.566021 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:29.566021 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:29.566021 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:29.566021 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:29.566021 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:29.566021 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:29.566021 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:29.566756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:29.566756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:29.566756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:29.566756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:29.566756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:29.566756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:29.566756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:29.566756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:29.566756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:29.566756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:29.566756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:28:29.566756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:28:29.571318 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 31/68] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:28:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:28:29.577934 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070eedc-02c0-4c02-9423-e4ef67b53821 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:29.578236 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070eedc-02c0-4c02-9423-e4ef67b53821 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:29.578566 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070eedc-02c0-4c02-9423-e4ef67b53821 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:29.579904 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 32/69] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:29.580720 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f070eedc-02c0-4c02-9423-e4ef67b53821 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:29.595544 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:29.679994 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f5b00-c458-4e70-826c-05929bbd73b0 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:29.697743 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f070eedc-02c0-4c02-9423-e4ef67b53821 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:29.715970 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b9fa6-50aa-46de-960d-9f961e691125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:29.716802 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b9fa6-50aa-46de-960d-9f961e691125 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:29.717090 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b9fa6-50aa-46de-960d-9f961e691125 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:29.717367 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b9fa6-50aa-46de-960d-9f961e691125 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:29.728780 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1b9fa6-50aa-46de-960d-9f961e691125 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:29.731972 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6fd514-77d5-481c-b0e4-82c5c146c9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:29.732647 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fd514-77d5-481c-b0e4-82c5c146c9d7 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:29.732942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fd514-77d5-481c-b0e4-82c5c146c9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:29.733228 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fd514-77d5-481c-b0e4-82c5c146c9d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:29.733987 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6fd514-77d5-481c-b0e4-82c5c146c9d7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:29.734305 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6fd514-77d5-481c-b0e4-82c5c146c9d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:29.735745 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6fd514-77d5-481c-b0e4-82c5c146c9d7 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:29.736480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-2c1b9fa6-50aa-46de-960d-9f961e691125 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:28:29.736480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:29.736480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:29.736480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:29.736480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:29.736480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:29.736480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:29.736480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:29.736480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:29.736480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:29.736480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:29.736480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:29.736480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:29.736480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:29.736480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:29.736480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:29.737428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:29.737428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:29.737428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:29.737428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:29.737428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:29.737428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:29.737428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:29.737428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:29.737428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:29.737428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:29.737428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:29.737428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:29.737428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:29.737428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:29.737428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:29.737428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:29.738335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:29.738335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:29.738335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:29.738335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:29.738335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:29.738335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:29.738335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:29.738335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:29.738335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:29.738335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:29.738335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:28:29.738335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:28:29.742582 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 38/70] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:28:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:28:29.743383 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b9fa6-50aa-46de-960d-9f961e691125 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:29.744742 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 33/71] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Jun 15 14:28:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:29.746524 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea5f81-b66d-4c3d-920d-50c5725d93b3 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:29.746815 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea5f81-b66d-4c3d-920d-50c5725d93b3 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:29.747133 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea5f81-b66d-4c3d-920d-50c5725d93b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:29.763221 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-f3ea5f81-b66d-4c3d-920d-50c5725d93b3 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:29.764434 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f3ea5f81-b66d-4c3d-920d-50c5725d93b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:29.816492 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:29.831132 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 34/72] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:29.838259 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221b23c-48e1-4aa9-a1f7-5e38e9dbd471 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:29.838682 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221b23c-48e1-4aa9-a1f7-5e38e9dbd471 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:29.838943 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221b23c-48e1-4aa9-a1f7-5e38e9dbd471 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:29.840412 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 35/73] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:29 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:29.845003 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221b23c-48e1-4aa9-a1f7-5e38e9dbd471 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:29.845125 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:29.854699 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b9fa6-50aa-46de-960d-9f961e691125 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:29.889410 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ee0f5-30fd-4d7b-ae8d-a31eb0795be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:29.890226 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ee0f5-30fd-4d7b-ae8d-a31eb0795be2 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:29.890517 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ee0f5-30fd-4d7b-ae8d-a31eb0795be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:29.890802 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ee0f5-30fd-4d7b-ae8d-a31eb0795be2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:29.901399 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ee0f5-30fd-4d7b-ae8d-a31eb0795be2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:29.909428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-111ee0f5-30fd-4d7b-ae8d-a31eb0795be2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:28:29.909428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:29.909428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:29.909428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:29.909428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:29.909428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:29.909428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:29.909428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:29.909428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:29.909428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:29.909428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:29.909428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:29.909428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:29.909428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:29.909428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:29.909428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:29.910312 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:29.910312 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:29.910312 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:29.910312 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:29.910312 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:29.910312 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:29.910312 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:29.910312 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:29.910312 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:29.910312 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:29.910312 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:29.910312 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:29.910312 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:29.910312 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:29.910312 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:29.910312 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:29.911405 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:29.911405 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:29.911405 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:29.911405 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:29.911405 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:29.911405 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:29.911405 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:29.911405 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:29.911405 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:29.911405 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:29.911405 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:28:29.911405 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:28:29.911405 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 39/74] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Jun 15 14:28:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:28:29.912165 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ee0f5-30fd-4d7b-ae8d-a31eb0795be2 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:29.945800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221b23c-48e1-4aa9-a1f7-5e38e9dbd471 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:29.978846 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f794c78-f8df-477d-ba88-1f0896f3d9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:29.979991 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f794c78-f8df-477d-ba88-1f0896f3d9e2 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:29.980275 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f794c78-f8df-477d-ba88-1f0896f3d9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:29.980546 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f794c78-f8df-477d-ba88-1f0896f3d9e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:29.981289 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f794c78-f8df-477d-ba88-1f0896f3d9e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:29.981595 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f794c78-f8df-477d-ba88-1f0896f3d9e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:29.982980 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f794c78-f8df-477d-ba88-1f0896f3d9e2 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:29.990848 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 36/75] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:28:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:29.992122 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292017fd-5800-4426-98f5-131b34049813 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:29.992419 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292017fd-5800-4426-98f5-131b34049813 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:29.992701 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292017fd-5800-4426-98f5-131b34049813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:30.009391 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-292017fd-5800-4426-98f5-131b34049813 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:30.010579 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-292017fd-5800-4426-98f5-131b34049813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:30.011428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ee0f5-30fd-4d7b-ae8d-a31eb0795be2 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:30.044445 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe22c4b-79ac-4c69-89f8-695b4fc282ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:30.045316 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe22c4b-79ac-4c69-89f8-695b4fc282ed None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:30.045603 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe22c4b-79ac-4c69-89f8-695b4fc282ed None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:30.045877 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe22c4b-79ac-4c69-89f8-695b4fc282ed None admin] PATH_INFO: `/v3/users/34d8aa4f47ae452592b3481bdb02e32a` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:30.056461 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:30.059968 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe22c4b-79ac-4c69-89f8-695b4fc282ed None admin] RBAC: Authorizing `identity:get_user(user_id=34d8aa4f47ae452592b3481bdb02e32a)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:30.061457 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe22c4b-79ac-4c69-89f8-695b4fc282ed None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:30.063183 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 40/76] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:29 2022] GET /identity/v3/users/34d8aa4f47ae452592b3481bdb02e32a => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:30.063958 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe22c4b-79ac-4c69-89f8-695b4fc282ed None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:30.076646 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:30.077862 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 37/77] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:30.078698 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292017fd-5800-4426-98f5-131b34049813 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:30.177033 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe22c4b-79ac-4c69-89f8-695b4fc282ed None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:30.198972 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292017fd-5800-4426-98f5-131b34049813 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:30.210927 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac6bbd-f8ed-4f29-81df-1fdf0365779a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:30.211767 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac6bbd-f8ed-4f29-81df-1fdf0365779a None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:30.212058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac6bbd-f8ed-4f29-81df-1fdf0365779a None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:30.212340 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac6bbd-f8ed-4f29-81df-1fdf0365779a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:30.222410 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac6bbd-f8ed-4f29-81df-1fdf0365779a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:30.230783 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-0fac6bbd-f8ed-4f29-81df-1fdf0365779a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:28:30.230783 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:30.230783 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:30.230783 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:30.230783 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:30.230783 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:30.230783 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:30.230783 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:30.230783 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:30.230783 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:30.230783 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:30.230783 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:30.230783 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:30.230783 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:30.230783 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:30.230783 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:30.231856 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:30.231856 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:30.231856 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:30.231856 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:30.231856 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:30.231856 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:30.231856 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:30.231856 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:30.231856 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:30.231856 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:30.231856 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:30.231856 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:30.231856 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:30.231856 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:30.231856 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:30.231856 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:30.232787 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:30.232787 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:30.232787 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:30.232787 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:30.232787 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:30.232787 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:30.232787 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:30.232787 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:30.232787 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:30.232787 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:30.232787 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:28:30.232787 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:28:30.233493 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa2461-eac0-4e4d-b11c-432946497f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:30.233493 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa2461-eac0-4e4d-b11c-432946497f03 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:30.233493 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa2461-eac0-4e4d-b11c-432946497f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:30.233493 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa2461-eac0-4e4d-b11c-432946497f03 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:30.234219 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa2461-eac0-4e4d-b11c-432946497f03 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:30.234533 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa2461-eac0-4e4d-b11c-432946497f03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:30.235072 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 41/78] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:28:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:28:30.235862 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac6bbd-f8ed-4f29-81df-1fdf0365779a None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:30.235952 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa2461-eac0-4e4d-b11c-432946497f03 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:30.248896 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 38/79] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:28:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:30.248896 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa2461-eac0-4e4d-b11c-432946497f03 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:30.288080 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:30.306088 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:30.336827 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac6bbd-f8ed-4f29-81df-1fdf0365779a None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:30.347818 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa2461-eac0-4e4d-b11c-432946497f03 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:30.370240 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7cde4b-dccb-4136-a106-4511ba7e970e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:30.371150 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cde4b-dccb-4136-a106-4511ba7e970e None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:30.371441 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cde4b-dccb-4136-a106-4511ba7e970e None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:30.371719 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cde4b-dccb-4136-a106-4511ba7e970e None admin] PATH_INFO: `/v3/users/a476d9b478cd40989e9e4e02fbc699e5` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:30.372926 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7cde4b-dccb-4136-a106-4511ba7e970e None admin] RBAC: Authorizing `identity:get_user(user_id=a476d9b478cd40989e9e4e02fbc699e5)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:30.374400 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7cde4b-dccb-4136-a106-4511ba7e970e None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:30.376100 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 42/80] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:30 2022] GET /identity/v3/users/a476d9b478cd40989e9e4e02fbc699e5 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:30.376847 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7cde4b-dccb-4136-a106-4511ba7e970e None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:30.381809 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01200f-7bda-42b8-8893-fac2abaa428e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:30.382466 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01200f-7bda-42b8-8893-fac2abaa428e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:30.382754 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01200f-7bda-42b8-8893-fac2abaa428e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:30.383066 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01200f-7bda-42b8-8893-fac2abaa428e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:30.450361 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01200f-7bda-42b8-8893-fac2abaa428e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:30.451865 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01200f-7bda-42b8-8893-fac2abaa428e None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:30.475299 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7cde4b-dccb-4136-a106-4511ba7e970e None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:30.493751 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:30.508510 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f86af-ddf7-4e88-917a-b0710f2be597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:30.509300 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f86af-ddf7-4e88-917a-b0710f2be597 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:30.509595 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f86af-ddf7-4e88-917a-b0710f2be597 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:30.509872 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f86af-ddf7-4e88-917a-b0710f2be597 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:30.511092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f86af-ddf7-4e88-917a-b0710f2be597 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:30.512540 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f86af-ddf7-4e88-917a-b0710f2be597 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:30.514163 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 43/81] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:28:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:30.514933 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f86af-ddf7-4e88-917a-b0710f2be597 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:30.523752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:30.551987 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 39/82] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:28:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:28:30.552926 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01200f-7bda-42b8-8893-fac2abaa428e None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:30.615051 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f86af-ddf7-4e88-917a-b0710f2be597 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:30.648177 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01200f-7bda-42b8-8893-fac2abaa428e None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:30.648322 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563ad3b-bf98-40da-9a4d-04afdbbd2b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:30.648969 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563ad3b-bf98-40da-9a4d-04afdbbd2b47 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:30.649262 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563ad3b-bf98-40da-9a4d-04afdbbd2b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:30.649551 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563ad3b-bf98-40da-9a4d-04afdbbd2b47 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:30.650302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563ad3b-bf98-40da-9a4d-04afdbbd2b47 None admin] RBAC: Adding query filter params (name=member) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:30.650619 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563ad3b-bf98-40da-9a4d-04afdbbd2b47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:30.652599 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563ad3b-bf98-40da-9a4d-04afdbbd2b47 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:30.660655 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 44/83] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Jun 15 14:28:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:30.661449 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563ad3b-bf98-40da-9a4d-04afdbbd2b47 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:30.687574 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973f3d7-9c2f-4c39-87bb-28f550a3a5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:30.688474 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973f3d7-9c2f-4c39-87bb-28f550a3a5d6 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:30.688762 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973f3d7-9c2f-4c39-87bb-28f550a3a5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:30.689038 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973f3d7-9c2f-4c39-87bb-28f550a3a5d6 None admin] PATH_INFO: `/v3/users/34d8aa4f47ae452592b3481bdb02e32a` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:30.690249 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973f3d7-9c2f-4c39-87bb-28f550a3a5d6 None admin] RBAC: Authorizing `identity:get_user(user_id=34d8aa4f47ae452592b3481bdb02e32a)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:30.691723 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973f3d7-9c2f-4c39-87bb-28f550a3a5d6 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:30.693354 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 40/84] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:30 2022] GET /identity/v3/users/34d8aa4f47ae452592b3481bdb02e32a => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:30.694099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973f3d7-9c2f-4c39-87bb-28f550a3a5d6 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:30.706033 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:30.728674 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:30.764338 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563ad3b-bf98-40da-9a4d-04afdbbd2b47 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:30.796349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973f3d7-9c2f-4c39-87bb-28f550a3a5d6 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:30.797693 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe42f70-8232-406b-a30f-aa90b02e513c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:30.798346 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe42f70-8232-406b-a30f-aa90b02e513c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:30.798638 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe42f70-8232-406b-a30f-aa90b02e513c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:30.798916 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe42f70-8232-406b-a30f-aa90b02e513c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:30.829092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568e0d95-07eb-4787-8ed8-f7a70096b2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:30.829755 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e0d95-07eb-4787-8ed8-f7a70096b2b9 None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:30.830039 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e0d95-07eb-4787-8ed8-f7a70096b2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:30.830325 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e0d95-07eb-4787-8ed8-f7a70096b2b9 None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:30.831062 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568e0d95-07eb-4787-8ed8-f7a70096b2b9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:30.832507 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568e0d95-07eb-4787-8ed8-f7a70096b2b9 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:30.852989 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 41/85] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Jun 15 14:28:30 2022] POST /identity/v3/users => generated 289 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:28:30.853787 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568e0d95-07eb-4787-8ed8-f7a70096b2b9 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:30.878564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe42f70-8232-406b-a30f-aa90b02e513c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:30.880562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe42f70-8232-406b-a30f-aa90b02e513c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:30.925811 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:30.963744 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:30.970242 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568e0d95-07eb-4787-8ed8-f7a70096b2b9 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:31.004231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 45/86] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:28:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:28:31.005104 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe42f70-8232-406b-a30f-aa90b02e513c None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:31.014595 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d15e4-02f4-4704-8931-8a53ef3b89ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:31.015299 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d15e4-02f4-4704-8931-8a53ef3b89ad None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:31.015582 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d15e4-02f4-4704-8931-8a53ef3b89ad None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:31.015861 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d15e4-02f4-4704-8931-8a53ef3b89ad None admin] PATH_INFO: `/v3/projects/8f586cc781db4f2b96f13e85c4851e5a` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:31.017318 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80d15e4-02f4-4704-8931-8a53ef3b89ad None admin] RBAC: Authorizing `identity:get_project(project_id=8f586cc781db4f2b96f13e85c4851e5a)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:31.018790 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80d15e4-02f4-4704-8931-8a53ef3b89ad None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:31.020698 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 42/87] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:28:30 2022] GET /identity/v3/projects/8f586cc781db4f2b96f13e85c4851e5a => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:31.021284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d15e4-02f4-4704-8931-8a53ef3b89ad None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:31.142380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d15e4-02f4-4704-8931-8a53ef3b89ad None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:31.142713 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe42f70-8232-406b-a30f-aa90b02e513c None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:31.189560 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ecf668-820d-4cb0-b59c-fea5867fcfbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:31.190232 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ecf668-820d-4cb0-b59c-fea5867fcfbc None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:31.190526 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ecf668-820d-4cb0-b59c-fea5867fcfbc None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:31.190809 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ecf668-820d-4cb0-b59c-fea5867fcfbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:31.192624 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be3974-9682-4ff3-abf1-3ea7e32c747d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:31.193725 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be3974-9682-4ff3-abf1-3ea7e32c747d None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:31.194199 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be3974-9682-4ff3-abf1-3ea7e32c747d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:31.194690 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be3974-9682-4ff3-abf1-3ea7e32c747d None admin] PATH_INFO: `/v3/users/34d8aa4f47ae452592b3481bdb02e32a` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:31.199638 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:31.200080 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be3974-9682-4ff3-abf1-3ea7e32c747d None admin] RBAC: Authorizing `identity:get_user(user_id=34d8aa4f47ae452592b3481bdb02e32a)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:31.200080 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be3974-9682-4ff3-abf1-3ea7e32c747d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:31.200080 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 46/88] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:31 2022] GET /identity/v3/users/34d8aa4f47ae452592b3481bdb02e32a => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:31.200080 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be3974-9682-4ff3-abf1-3ea7e32c747d None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:31.236442 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:31.267966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ecf668-820d-4cb0-b59c-fea5867fcfbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:31.269499 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ecf668-820d-4cb0-b59c-fea5867fcfbc None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:31.303767 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be3974-9682-4ff3-abf1-3ea7e32c747d None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:31.340198 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1944c27f-ef70-4386-b044-2ceb0205f1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:31.341163 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944c27f-ef70-4386-b044-2ceb0205f1ed None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:31.341443 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944c27f-ef70-4386-b044-2ceb0205f1ed None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:31.341727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944c27f-ef70-4386-b044-2ceb0205f1ed None admin] PATH_INFO: `/v3/projects/8f586cc781db4f2b96f13e85c4851e5a` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:31.346439 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1944c27f-ef70-4386-b044-2ceb0205f1ed None admin] RBAC: Authorizing `identity:get_project(project_id=8f586cc781db4f2b96f13e85c4851e5a)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:31.346439 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1944c27f-ef70-4386-b044-2ceb0205f1ed None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:31.346439 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 47/89] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:28:31 2022] GET /identity/v3/projects/8f586cc781db4f2b96f13e85c4851e5a => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:31.350105 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1944c27f-ef70-4386-b044-2ceb0205f1ed None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:31.378852 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 43/90] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:28:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:28:31.383721 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ecf668-820d-4cb0-b59c-fea5867fcfbc None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:31.421344 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:31.456555 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1944c27f-ef70-4386-b044-2ceb0205f1ed None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:31.459143 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:31.495213 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107feeaa-48aa-40bd-95b6-8338ec7137f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:31.495995 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107feeaa-48aa-40bd-95b6-8338ec7137f0 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:31.496285 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107feeaa-48aa-40bd-95b6-8338ec7137f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:31.496564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107feeaa-48aa-40bd-95b6-8338ec7137f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:31.497507 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107feeaa-48aa-40bd-95b6-8338ec7137f0 None admin] RBAC: Adding query filter params (role.id=ce0de9b9c9054826b569db895942ad26, scope.project.id=8f586cc781db4f2b96f13e85c4851e5a, user.id=34d8aa4f47ae452592b3481bdb02e32a) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:31.497826 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107feeaa-48aa-40bd-95b6-8338ec7137f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:31.499304 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107feeaa-48aa-40bd-95b6-8338ec7137f0 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:31.500231 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ecf668-820d-4cb0-b59c-fea5867fcfbc None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:31.505308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 48/91] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:28:31 2022] GET /identity/v3/role_assignments?user.id=34d8aa4f47ae452592b3481bdb02e32a&scope.project.id=8f586cc781db4f2b96f13e85c4851e5a&role.id=ce0de9b9c9054826b569db895942ad26 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:31.506167 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107feeaa-48aa-40bd-95b6-8338ec7137f0 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:31.568153 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83345a00-16be-44ad-bd90-9c6b483b83c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:31.569082 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83345a00-16be-44ad-bd90-9c6b483b83c5 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:31.569509 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83345a00-16be-44ad-bd90-9c6b483b83c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:31.569927 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83345a00-16be-44ad-bd90-9c6b483b83c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:31.625335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107feeaa-48aa-40bd-95b6-8338ec7137f0 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:31.701462 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83345a00-16be-44ad-bd90-9c6b483b83c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:31.704747 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83345a00-16be-44ad-bd90-9c6b483b83c5 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:31.744391 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37306adf-cd7d-40a8-8ba7-9644cde6c971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:31.746030 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37306adf-cd7d-40a8-8ba7-9644cde6c971 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:31.746322 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37306adf-cd7d-40a8-8ba7-9644cde6c971 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:31.746602 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37306adf-cd7d-40a8-8ba7-9644cde6c971 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:31.747485 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37306adf-cd7d-40a8-8ba7-9644cde6c971 None admin] RBAC: Adding query filter params (role.id=ed1cfa2fcf724a4d84f5acea100930c7, scope.project.id=8f586cc781db4f2b96f13e85c4851e5a, user.id=a476d9b478cd40989e9e4e02fbc699e5) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:31.747803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37306adf-cd7d-40a8-8ba7-9644cde6c971 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:31.749266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37306adf-cd7d-40a8-8ba7-9644cde6c971 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:31.751223 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:31.757208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 49/92] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:28:31 2022] GET /identity/v3/role_assignments?user.id=a476d9b478cd40989e9e4e02fbc699e5&scope.project.id=8f586cc781db4f2b96f13e85c4851e5a&role.id=ed1cfa2fcf724a4d84f5acea100930c7 => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:31.764077 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37306adf-cd7d-40a8-8ba7-9644cde6c971 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:31.823049 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:31.836144 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 44/93] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:28:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:28:31.843076 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83345a00-16be-44ad-bd90-9c6b483b83c5 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:31.978365 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37306adf-cd7d-40a8-8ba7-9644cde6c971 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:31.993338 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83345a00-16be-44ad-bd90-9c6b483b83c5 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:32.035321 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7089a9f2-9711-4b2b-a6c2-9dce8f41701b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:32.036566 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089a9f2-9711-4b2b-a6c2-9dce8f41701b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:32.037237 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089a9f2-9711-4b2b-a6c2-9dce8f41701b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:32.038507 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089a9f2-9711-4b2b-a6c2-9dce8f41701b None admin] PATH_INFO: `/v3/projects/3c26a382a0344c9580d9217b0f98e064` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:32.040803 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7089a9f2-9711-4b2b-a6c2-9dce8f41701b None admin] RBAC: Authorizing `identity:get_project(project_id=3c26a382a0344c9580d9217b0f98e064)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:32.043907 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7089a9f2-9711-4b2b-a6c2-9dce8f41701b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:32.046841 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 45/94] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:28:31 2022] GET /identity/v3/projects/3c26a382a0344c9580d9217b0f98e064 => generated 312 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:32.052917 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7089a9f2-9711-4b2b-a6c2-9dce8f41701b None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:32.057298 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea22362-e931-4370-a621-6feea45e74b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:32.059091 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea22362-e931-4370-a621-6feea45e74b8 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:32.059584 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea22362-e931-4370-a621-6feea45e74b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:32.060088 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea22362-e931-4370-a621-6feea45e74b8 None admin] PATH_INFO: `/v3/projects/8f586cc781db4f2b96f13e85c4851e5a` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:32.061593 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:32.064169 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea22362-e931-4370-a621-6feea45e74b8 None admin] RBAC: Authorizing `identity:get_project(project_id=8f586cc781db4f2b96f13e85c4851e5a)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:32.065889 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea22362-e931-4370-a621-6feea45e74b8 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:32.067957 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 50/95] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:28:31 2022] GET /identity/v3/projects/8f586cc781db4f2b96f13e85c4851e5a => generated 298 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:32.076671 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea22362-e931-4370-a621-6feea45e74b8 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:32.266241 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7089a9f2-9711-4b2b-a6c2-9dce8f41701b None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:32.271301 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:32.282749 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea22362-e931-4370-a621-6feea45e74b8 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:32.339734 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab67a57-f002-4677-84eb-430f483023ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:32.340441 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab67a57-f002-4677-84eb-430f483023ba None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:32.340735 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab67a57-f002-4677-84eb-430f483023ba None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:32.341017 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab67a57-f002-4677-84eb-430f483023ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:32.342073 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab67a57-f002-4677-84eb-430f483023ba None admin] RBAC: Adding query filter params (role.id=ce0de9b9c9054826b569db895942ad26, scope.project.id=3c26a382a0344c9580d9217b0f98e064, user.id=34d8aa4f47ae452592b3481bdb02e32a) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:32.342395 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab67a57-f002-4677-84eb-430f483023ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:32.344251 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab67a57-f002-4677-84eb-430f483023ba None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:32.353143 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64652355-9699-462b-a437-3ad9ed3ec903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:32.354288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64652355-9699-462b-a437-3ad9ed3ec903 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:32.355321 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64652355-9699-462b-a437-3ad9ed3ec903 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:32.355772 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64652355-9699-462b-a437-3ad9ed3ec903 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:32.356222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 46/96] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:28:32 2022] GET /identity/v3/role_assignments?user.id=34d8aa4f47ae452592b3481bdb02e32a&scope.project.id=3c26a382a0344c9580d9217b0f98e064&role.id=ce0de9b9c9054826b569db895942ad26 => generated 259 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:32.357002 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64652355-9699-462b-a437-3ad9ed3ec903 None admin] RBAC: Adding query filter params (role.id=6b6353b49c6a41b1a157f8997fc52d9a, scope.project.id=8f586cc781db4f2b96f13e85c4851e5a, user.id=34d8aa4f47ae452592b3481bdb02e32a) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:32.357492 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64652355-9699-462b-a437-3ad9ed3ec903 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:32.359200 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64652355-9699-462b-a437-3ad9ed3ec903 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:32.372444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 51/97] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:28:32 2022] GET /identity/v3/role_assignments?user.id=34d8aa4f47ae452592b3481bdb02e32a&scope.project.id=8f586cc781db4f2b96f13e85c4851e5a&role.id=6b6353b49c6a41b1a157f8997fc52d9a => generated 259 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:33.519764 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd684532-8011-4667-9037-f0af0d0ef674 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:33.520273 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd684532-8011-4667-9037-f0af0d0ef674 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:33.520345 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd684532-8011-4667-9037-f0af0d0ef674 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:33.521710 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 47/98] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:33.536310 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a2b38-2beb-4b24-ac47-1c3708f9e61f None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:33.537191 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a2b38-2beb-4b24-ac47-1c3708f9e61f None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:33.537675 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a2b38-2beb-4b24-ac47-1c3708f9e61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:33.558858 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-406a2b38-2beb-4b24-ac47-1c3708f9e61f None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:33.560365 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-406a2b38-2beb-4b24-ac47-1c3708f9e61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:33.647123 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 52/99] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:33.687288 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd684532-8011-4667-9037-f0af0d0ef674 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:33.782658 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:33.832039 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd684532-8011-4667-9037-f0af0d0ef674 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:33.882052 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb76bc1d-a085-4b92-96c0-806bf1e3f26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:33.882856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76bc1d-a085-4b92-96c0-806bf1e3f26b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:33.883174 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76bc1d-a085-4b92-96c0-806bf1e3f26b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:33.883451 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76bc1d-a085-4b92-96c0-806bf1e3f26b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:33.897180 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb76bc1d-a085-4b92-96c0-806bf1e3f26b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:33.907488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-eb76bc1d-a085-4b92-96c0-806bf1e3f26b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:28:33.907488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:33.907488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:33.907488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:33.907488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:33.907488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:33.907488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:33.907488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:33.907488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:33.907488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:33.907488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:33.907488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:33.907488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:33.907488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:33.907488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:33.907488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:33.908340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:33.908340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:33.908340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:33.908340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:33.908340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:33.908340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:33.908340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:33.908340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:33.908340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:33.908340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:33.908340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:33.908340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:33.908340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:33.908340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:33.908340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:33.908340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:33.909068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:33.909068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:33.909068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:33.909068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:33.909068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:33.909068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:33.909068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:33.909068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:33.909068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:33.909068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:33.909068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:28:33.909068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:28:33.915332 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 48/100] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:28:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:28:33.916109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a2b38-2beb-4b24-ac47-1c3708f9e61f None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:34.113593 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1c6bd-0b0c-4d29-8545-cc2fd4114ef0 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:34.113955 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1c6bd-0b0c-4d29-8545-cc2fd4114ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:34.114152 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1c6bd-0b0c-4d29-8545-cc2fd4114ef0 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:34.114244 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:34.117097 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 49/101] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:28:34.121879 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a2b38-2beb-4b24-ac47-1c3708f9e61f None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:34.128133 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66991a2b-3708-4ae3-b79e-5af9829355d6 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:34.128421 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66991a2b-3708-4ae3-b79e-5af9829355d6 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:34.128948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66991a2b-3708-4ae3-b79e-5af9829355d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:34.168150 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-66991a2b-3708-4ae3-b79e-5af9829355d6 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:34.169351 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-66991a2b-3708-4ae3-b79e-5af9829355d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:34.179318 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c0f8aa-c260-4bd7-8ff9-9a1a1167c137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:34.180005 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0f8aa-c260-4bd7-8ff9-9a1a1167c137 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:34.180296 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0f8aa-c260-4bd7-8ff9-9a1a1167c137 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:34.180578 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0f8aa-c260-4bd7-8ff9-9a1a1167c137 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:34.181351 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c0f8aa-c260-4bd7-8ff9-9a1a1167c137 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:34.181667 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c0f8aa-c260-4bd7-8ff9-9a1a1167c137 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:34.183647 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c0f8aa-c260-4bd7-8ff9-9a1a1167c137 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:34.198296 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 53/102] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Jun 15 14:28:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:34.204320 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c0f8aa-c260-4bd7-8ff9-9a1a1167c137 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:34.303016 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 50/103] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:34.331852 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c0f8aa-c260-4bd7-8ff9-9a1a1167c137 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:34.351034 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66991a2b-3708-4ae3-b79e-5af9829355d6 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:34.376281 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b9e6e-3f39-4eed-a84f-e72b0aaaa172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:34.377480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b9e6e-3f39-4eed-a84f-e72b0aaaa172 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:34.377959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b9e6e-3f39-4eed-a84f-e72b0aaaa172 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:34.378424 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b9e6e-3f39-4eed-a84f-e72b0aaaa172 None admin] PATH_INFO: `/v3/users/4e1d412ac8d84b70a22ec80f41fded6c` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:34.402352 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001b9e6e-3f39-4eed-a84f-e72b0aaaa172 None admin] RBAC: Authorizing `identity:get_user(user_id=4e1d412ac8d84b70a22ec80f41fded6c)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:34.404271 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001b9e6e-3f39-4eed-a84f-e72b0aaaa172 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:34.406232 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 54/104] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:34 2022] GET /identity/v3/users/4e1d412ac8d84b70a22ec80f41fded6c => generated 289 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:34.411549 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b9e6e-3f39-4eed-a84f-e72b0aaaa172 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:34.447174 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:34.483174 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:34.597297 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66991a2b-3708-4ae3-b79e-5af9829355d6 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:34.626576 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b9e6e-3f39-4eed-a84f-e72b0aaaa172 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:34.679175 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c238a7-ea67-4fb3-890e-a0f718801529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:34.680095 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c238a7-ea67-4fb3-890e-a0f718801529 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:34.680381 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c238a7-ea67-4fb3-890e-a0f718801529 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:34.680658 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c238a7-ea67-4fb3-890e-a0f718801529 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:34.688012 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c238a7-ea67-4fb3-890e-a0f718801529 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:34.689512 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c238a7-ea67-4fb3-890e-a0f718801529 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:34.691273 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 51/105] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:28:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:34.702188 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda868ff-6719-4ecc-a21a-36b25245733c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:34.702712 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c238a7-ea67-4fb3-890e-a0f718801529 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:34.702892 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda868ff-6719-4ecc-a21a-36b25245733c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:34.703192 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda868ff-6719-4ecc-a21a-36b25245733c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:34.703479 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda868ff-6719-4ecc-a21a-36b25245733c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:34.859029 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda868ff-6719-4ecc-a21a-36b25245733c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:34.860655 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda868ff-6719-4ecc-a21a-36b25245733c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:34.877455 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:34.895010 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c238a7-ea67-4fb3-890e-a0f718801529 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:34.928568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:34.942545 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d42aa7b-64e3-44ab-8d82-d382b214b143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:34.943237 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42aa7b-64e3-44ab-8d82-d382b214b143 None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:34.943520 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42aa7b-64e3-44ab-8d82-d382b214b143 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:34.943796 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42aa7b-64e3-44ab-8d82-d382b214b143 None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:34.944518 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d42aa7b-64e3-44ab-8d82-d382b214b143 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:34.945954 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d42aa7b-64e3-44ab-8d82-d382b214b143 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:34.975325 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 52/106] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Jun 15 14:28:34 2022] POST /identity/v3/projects => generated 302 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:28:34.977245 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848f705-29a7-4908-bd9e-18fa36a2c8d3 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:34.977717 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848f705-29a7-4908-bd9e-18fa36a2c8d3 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:34.978294 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848f705-29a7-4908-bd9e-18fa36a2c8d3 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:34.979929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 53/107] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:34.980717 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 55/108] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:28:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 14:28:34.981909 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba197cf-a321-4680-b18c-6e021315e1a4 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:34.982494 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba197cf-a321-4680-b18c-6e021315e1a4 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:34.982949 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba197cf-a321-4680-b18c-6e021315e1a4 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:34.984455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 54/109] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:34.988645 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0351bf-8f6e-488f-acb9-f101dc0acc58 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:34.988645 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0351bf-8f6e-488f-acb9-f101dc0acc58 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:34.988645 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0351bf-8f6e-488f-acb9-f101dc0acc58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:34.990333 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba197cf-a321-4680-b18c-6e021315e1a4 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:35.005966 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-6d0351bf-8f6e-488f-acb9-f101dc0acc58 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:35.007341 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6d0351bf-8f6e-488f-acb9-f101dc0acc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:35.084630 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 56/110] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:35.085850 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af247ed8-6147-47d5-8197-ec3df834e6a0 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:35.086561 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af247ed8-6147-47d5-8197-ec3df834e6a0 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:35.086714 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af247ed8-6147-47d5-8197-ec3df834e6a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:35.104648 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-af247ed8-6147-47d5-8197-ec3df834e6a0 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:35.106065 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-af247ed8-6147-47d5-8197-ec3df834e6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:35.112848 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba197cf-a321-4680-b18c-6e021315e1a4 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:35.154373 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ede1ae-2380-4aba-aa49-2e46b7c5bd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:35.155685 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ede1ae-2380-4aba-aa49-2e46b7c5bd0a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:35.156189 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ede1ae-2380-4aba-aa49-2e46b7c5bd0a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:35.156660 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ede1ae-2380-4aba-aa49-2e46b7c5bd0a None admin] PATH_INFO: `/v3/projects/8f586cc781db4f2b96f13e85c4851e5a` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:35.158158 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ede1ae-2380-4aba-aa49-2e46b7c5bd0a None admin] RBAC: Authorizing `identity:get_project(project_id=8f586cc781db4f2b96f13e85c4851e5a)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:35.159141 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:35.161698 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ede1ae-2380-4aba-aa49-2e46b7c5bd0a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:35.164225 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 55/111] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:28:34 2022] GET /identity/v3/projects/8f586cc781db4f2b96f13e85c4851e5a => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:35.165352 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ede1ae-2380-4aba-aa49-2e46b7c5bd0a None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:35.214580 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 57/112] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:35.222821 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af247ed8-6147-47d5-8197-ec3df834e6a0 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:35.233779 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:35.331770 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af247ed8-6147-47d5-8197-ec3df834e6a0 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:35.341142 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ede1ae-2380-4aba-aa49-2e46b7c5bd0a None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:35.374159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f5542-9e1f-4717-a1f6-5a7f50797843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:35.375018 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f5542-9e1f-4717-a1f6-5a7f50797843 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:35.375334 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f5542-9e1f-4717-a1f6-5a7f50797843 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:35.375628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f5542-9e1f-4717-a1f6-5a7f50797843 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:35.376482 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f5542-9e1f-4717-a1f6-5a7f50797843 None admin] RBAC: Adding query filter params (role.id=30e8990a9ea24c4398bbe057f41400bb, scope.project.id=8f586cc781db4f2b96f13e85c4851e5a, user.id=4e1d412ac8d84b70a22ec80f41fded6c) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:35.376801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f5542-9e1f-4717-a1f6-5a7f50797843 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:35.378313 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f5542-9e1f-4717-a1f6-5a7f50797843 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:35.381933 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda0a206-ce9d-4f84-82eb-d33e98dc31dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:35.382951 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0a206-ce9d-4f84-82eb-d33e98dc31dd None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:35.383484 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0a206-ce9d-4f84-82eb-d33e98dc31dd None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:35.383934 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0a206-ce9d-4f84-82eb-d33e98dc31dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:35.384133 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 58/113] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:28:35 2022] GET /identity/v3/role_assignments?user.id=4e1d412ac8d84b70a22ec80f41fded6c&scope.project.id=8f586cc781db4f2b96f13e85c4851e5a&role.id=30e8990a9ea24c4398bbe057f41400bb => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:35.385090 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f5542-9e1f-4717-a1f6-5a7f50797843 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:35.398654 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda0a206-ce9d-4f84-82eb-d33e98dc31dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:35.407478 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-bda0a206-ce9d-4f84-82eb-d33e98dc31dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:28:35.407478 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:35.407478 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:35.407478 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:35.407478 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:35.407478 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:35.407478 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:35.407478 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:35.407478 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:35.407478 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:35.407478 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:35.407478 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:35.407478 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:35.407478 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:35.407478 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:35.407478 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:35.408528 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:35.408528 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:35.408528 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:35.408528 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:35.408528 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:35.408528 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:35.408528 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:35.408528 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:35.408528 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:35.408528 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:35.408528 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:35.408528 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:35.408528 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:35.408528 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:35.408528 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:35.408528 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:35.409493 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:35.409493 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:35.409493 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:35.409493 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:35.409493 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:35.409493 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:35.409493 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:35.409493 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:35.409493 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:35.409493 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:35.409493 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:28:35.409493 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:28:35.410946 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 56/114] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:28:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:28:35.412750 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c5bbe-dd06-4335-a37b-0d91e7de7198 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:35.413120 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c5bbe-dd06-4335-a37b-0d91e7de7198 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:35.413477 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c5bbe-dd06-4335-a37b-0d91e7de7198 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:35.414879 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 57/115] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:35.420255 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c5bbe-dd06-4335-a37b-0d91e7de7198 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:35.439862 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:35.549051 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f5542-9e1f-4717-a1f6-5a7f50797843 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:35.608354 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc5623-16ba-4f6b-93a3-e56683bad6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:35.609175 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc5623-16ba-4f6b-93a3-e56683bad6ad None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:35.609465 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc5623-16ba-4f6b-93a3-e56683bad6ad None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:35.609746 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc5623-16ba-4f6b-93a3-e56683bad6ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:35.620991 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c5bbe-dd06-4335-a37b-0d91e7de7198 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:35.624526 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbc5623-16ba-4f6b-93a3-e56683bad6ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:35.626090 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:35.635050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-9fbc5623-16ba-4f6b-93a3-e56683bad6ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:28:35.635050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:35.635050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:35.635050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:35.635050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:35.635050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:35.635050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:35.635050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:35.635050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:35.635050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:35.635050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:35.635050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:35.635050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:35.635050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:35.635050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:35.635050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:35.636778 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:35.636778 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:35.636778 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:35.636778 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:35.636778 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:35.636778 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:35.636778 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:35.636778 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:35.636778 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:35.636778 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:35.636778 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:35.636778 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:35.636778 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:35.636778 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:35.636778 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:35.636778 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:35.637534 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:35.637534 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:35.637534 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:35.637534 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:35.637534 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:35.637534 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:35.637534 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:35.637534 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:35.637534 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:35.637534 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:35.637534 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:28:35.637534 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:28:35.639985 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 59/116] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:28:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:28:35.641466 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2d26d-483a-47f5-90f6-d7b471dc58c0 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:35.642213 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2d26d-483a-47f5-90f6-d7b471dc58c0 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:35.642661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2d26d-483a-47f5-90f6-d7b471dc58c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:35.662009 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-22c2d26d-483a-47f5-90f6-d7b471dc58c0 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:35.663560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-22c2d26d-483a-47f5-90f6-d7b471dc58c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:35.670663 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695992a-e075-41ff-9946-475bdb3273ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:35.671431 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695992a-e075-41ff-9946-475bdb3273ba None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:35.671732 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695992a-e075-41ff-9946-475bdb3273ba None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:35.672020 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695992a-e075-41ff-9946-475bdb3273ba None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:35.673341 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695992a-e075-41ff-9946-475bdb3273ba None admin] RBAC: Adding query filter params (name=member) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:35.673681 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695992a-e075-41ff-9946-475bdb3273ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:35.683076 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695992a-e075-41ff-9946-475bdb3273ba None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:35.686979 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 58/117] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Jun 15 14:28:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:35.687695 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695992a-e075-41ff-9946-475bdb3273ba None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:35.804541 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 60/118] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:35.805409 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2d26d-483a-47f5-90f6-d7b471dc58c0 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:35.847583 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:35.854223 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695992a-e075-41ff-9946-475bdb3273ba None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:35.898181 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002c15d-5f39-482f-b1ff-b2ecaf563797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:35.898854 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002c15d-5f39-482f-b1ff-b2ecaf563797 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:35.899172 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002c15d-5f39-482f-b1ff-b2ecaf563797 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:35.899451 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002c15d-5f39-482f-b1ff-b2ecaf563797 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:35.900211 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c002c15d-5f39-482f-b1ff-b2ecaf563797 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:35.900517 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c002c15d-5f39-482f-b1ff-b2ecaf563797 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:35.902899 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c002c15d-5f39-482f-b1ff-b2ecaf563797 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:35.913951 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 59/119] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:28:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:35.915199 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192fd96-e39f-43bc-8c1f-d998285b59f2 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:35.915489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192fd96-e39f-43bc-8c1f-d998285b59f2 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:35.915766 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192fd96-e39f-43bc-8c1f-d998285b59f2 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:35.917474 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 60/120] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:35.918178 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f192fd96-e39f-43bc-8c1f-d998285b59f2 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:35.935213 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2d26d-483a-47f5-90f6-d7b471dc58c0 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:35.951865 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:35.983423 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5bad85-10df-4a62-b269-e5211401d486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:35.984359 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bad85-10df-4a62-b269-e5211401d486 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:35.985170 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bad85-10df-4a62-b269-e5211401d486 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:35.985515 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bad85-10df-4a62-b269-e5211401d486 None admin] PATH_INFO: `/v3/users/34d8aa4f47ae452592b3481bdb02e32a` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:35.987111 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5bad85-10df-4a62-b269-e5211401d486 None admin] RBAC: Authorizing `identity:get_user(user_id=34d8aa4f47ae452592b3481bdb02e32a)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:35.989403 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5bad85-10df-4a62-b269-e5211401d486 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:35.996057 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 61/121] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:35 2022] GET /identity/v3/users/34d8aa4f47ae452592b3481bdb02e32a => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:35.996057 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5bad85-10df-4a62-b269-e5211401d486 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:36.043107 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f192fd96-e39f-43bc-8c1f-d998285b59f2 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:36.080456 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5380c0f-7cc2-41fa-8572-7b6b4948a530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:36.082137 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5380c0f-7cc2-41fa-8572-7b6b4948a530 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:36.082691 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5380c0f-7cc2-41fa-8572-7b6b4948a530 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:36.083142 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5380c0f-7cc2-41fa-8572-7b6b4948a530 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:36.101389 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5380c0f-7cc2-41fa-8572-7b6b4948a530 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:36.109941 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-b5380c0f-7cc2-41fa-8572-7b6b4948a530 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:28:36.109941 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:36.109941 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:36.109941 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:36.109941 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:36.109941 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:36.109941 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:36.109941 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:36.109941 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:36.109941 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:36.109941 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:36.109941 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:36.109941 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:36.109941 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:36.109941 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:36.109941 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:36.110967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:36.110967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:36.110967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:36.110967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:36.110967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:36.110967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:36.110967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:36.110967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:36.110967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:36.110967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:36.110967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:36.110967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:36.110967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:36.110967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:36.110967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:36.110967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:36.112050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:36.112050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:36.112050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:36.112050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:36.112050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:36.112050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:36.112050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:36.112050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:36.112050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:36.112050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:36.112050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:28:36.112050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:28:36.112050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 61/122] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:28:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:28:36.112821 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e5b151-1a91-4bfc-a6a4-79e263cdf72f None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:36.112821 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e5b151-1a91-4bfc-a6a4-79e263cdf72f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:36.113088 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e5b151-1a91-4bfc-a6a4-79e263cdf72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:36.121918 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5bad85-10df-4a62-b269-e5211401d486 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:36.123982 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:36.138664 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-d3e5b151-1a91-4bfc-a6a4-79e263cdf72f None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:36.139959 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d3e5b151-1a91-4bfc-a6a4-79e263cdf72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:36.160690 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57f3776-d939-4542-b3c6-9feba3d56cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:36.161919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f3776-d939-4542-b3c6-9feba3d56cbe None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:36.162378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f3776-d939-4542-b3c6-9feba3d56cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:36.162822 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f3776-d939-4542-b3c6-9feba3d56cbe None admin] PATH_INFO: `/v3/users/a476d9b478cd40989e9e4e02fbc699e5` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:36.164307 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57f3776-d939-4542-b3c6-9feba3d56cbe None admin] RBAC: Authorizing `identity:get_user(user_id=a476d9b478cd40989e9e4e02fbc699e5)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:36.165974 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57f3776-d939-4542-b3c6-9feba3d56cbe None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:36.167887 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 62/123] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:35 2022] GET /identity/v3/users/a476d9b478cd40989e9e4e02fbc699e5 => generated 247 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:36.168938 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57f3776-d939-4542-b3c6-9feba3d56cbe None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:36.228571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 62/124] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:36.235163 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e5b151-1a91-4bfc-a6a4-79e263cdf72f None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:36.253581 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:36.300271 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57f3776-d939-4542-b3c6-9feba3d56cbe None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:36.338918 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb36454-c911-4cab-9d29-cccea37273aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:36.340117 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb36454-c911-4cab-9d29-cccea37273aa None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:36.340574 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb36454-c911-4cab-9d29-cccea37273aa None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:36.341039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb36454-c911-4cab-9d29-cccea37273aa None admin] PATH_INFO: `/v3/projects/8f586cc781db4f2b96f13e85c4851e5a` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:36.342412 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb36454-c911-4cab-9d29-cccea37273aa None admin] RBAC: Authorizing `identity:get_project(project_id=8f586cc781db4f2b96f13e85c4851e5a)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:36.344108 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb36454-c911-4cab-9d29-cccea37273aa None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:36.346011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 63/125] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:28:36 2022] GET /identity/v3/projects/8f586cc781db4f2b96f13e85c4851e5a => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:36.347116 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb36454-c911-4cab-9d29-cccea37273aa None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:36.354943 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e5b151-1a91-4bfc-a6a4-79e263cdf72f None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:36.380393 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:36.403087 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb7371-d858-42bb-875b-eaa58670506d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:36.403741 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb7371-d858-42bb-875b-eaa58670506d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:36.404027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb7371-d858-42bb-875b-eaa58670506d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:36.404305 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb7371-d858-42bb-875b-eaa58670506d None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:36.405051 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb7371-d858-42bb-875b-eaa58670506d None admin] RBAC: Adding query filter params (name=member) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:36.405357 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb7371-d858-42bb-875b-eaa58670506d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:36.407304 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb7371-d858-42bb-875b-eaa58670506d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:36.418663 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 63/126] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Jun 15 14:28:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:36.419435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb7371-d858-42bb-875b-eaa58670506d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:36.461848 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb36454-c911-4cab-9d29-cccea37273aa None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:36.501831 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566150fd-9c72-4e71-a67c-e8f49a3ed244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:36.502709 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566150fd-9c72-4e71-a67c-e8f49a3ed244 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:36.503066 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566150fd-9c72-4e71-a67c-e8f49a3ed244 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:36.503526 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566150fd-9c72-4e71-a67c-e8f49a3ed244 None admin] PATH_INFO: `/v3/projects/8f586cc781db4f2b96f13e85c4851e5a` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:36.505260 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566150fd-9c72-4e71-a67c-e8f49a3ed244 None admin] RBAC: Authorizing `identity:get_project(project_id=8f586cc781db4f2b96f13e85c4851e5a)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:36.506819 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566150fd-9c72-4e71-a67c-e8f49a3ed244 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:36.508828 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 64/127] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:28:36 2022] GET /identity/v3/projects/8f586cc781db4f2b96f13e85c4851e5a => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:36.510165 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566150fd-9c72-4e71-a67c-e8f49a3ed244 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:36.528360 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:36.550588 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb7371-d858-42bb-875b-eaa58670506d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:36.593620 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18892f-66bf-4f37-843c-c72c37e8c86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:36.594585 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18892f-66bf-4f37-843c-c72c37e8c86a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:36.594880 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18892f-66bf-4f37-843c-c72c37e8c86a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:36.595239 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18892f-66bf-4f37-843c-c72c37e8c86a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:36.608606 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e18892f-66bf-4f37-843c-c72c37e8c86a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:36.619966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-9e18892f-66bf-4f37-843c-c72c37e8c86a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:28:36.619966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:36.619966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:36.619966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:36.619966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:36.619966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:36.619966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:36.619966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:36.619966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:36.619966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:36.619966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:36.619966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:36.619966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:36.619966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:36.619966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:36.619966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:36.622036 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:36.622036 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:36.622036 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:36.622036 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:36.622036 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:36.622036 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:36.622036 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:36.622036 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:36.622036 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:36.622036 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:36.622036 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:36.622036 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:36.622036 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:36.622036 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:36.622036 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:36.622036 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:36.622938 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:36.622938 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:36.622938 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:36.622938 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:36.622938 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:36.622938 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:36.622938 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:36.622938 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:36.622938 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:36.622938 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:36.622938 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:28:36.622938 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:28:36.625874 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 64/128] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Jun 15 14:28:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:28:36.626931 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18892f-66bf-4f37-843c-c72c37e8c86a None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:36.648492 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:36.652877 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566150fd-9c72-4e71-a67c-e8f49a3ed244 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:36.691611 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a12b0-e0f3-427f-9c9d-986642ca17c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:36.692532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a12b0-e0f3-427f-9c9d-986642ca17c6 None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:36.692823 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a12b0-e0f3-427f-9c9d-986642ca17c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:36.693133 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a12b0-e0f3-427f-9c9d-986642ca17c6 None admin] PATH_INFO: `/v3/projects/8f586cc781db4f2b96f13e85c4851e5a/users/34d8aa4f47ae452592b3481bdb02e32a/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:36.695251 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a12b0-e0f3-427f-9c9d-986642ca17c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=8f586cc781db4f2b96f13e85c4851e5a, user_id=34d8aa4f47ae452592b3481bdb02e32a, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:36.696939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a12b0-e0f3-427f-9c9d-986642ca17c6 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:36.719800 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 65/129] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Jun 15 14:28:36 2022] PUT /identity/v3/projects/8f586cc781db4f2b96f13e85c4851e5a/users/34d8aa4f47ae452592b3481bdb02e32a/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:28:36.720619 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a12b0-e0f3-427f-9c9d-986642ca17c6 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:36.774169 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18892f-66bf-4f37-843c-c72c37e8c86a None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:36.818195 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c9248-3373-4014-a10e-79e4936dd332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:36.819491 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c9248-3373-4014-a10e-79e4936dd332 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:36.819935 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c9248-3373-4014-a10e-79e4936dd332 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:36.820365 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c9248-3373-4014-a10e-79e4936dd332 None admin] PATH_INFO: `/v3/users/34d8aa4f47ae452592b3481bdb02e32a` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:36.821899 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c9248-3373-4014-a10e-79e4936dd332 None admin] RBAC: Authorizing `identity:get_user(user_id=34d8aa4f47ae452592b3481bdb02e32a)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:36.823700 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c9248-3373-4014-a10e-79e4936dd332 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:36.827040 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 65/130] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:36 2022] GET /identity/v3/users/34d8aa4f47ae452592b3481bdb02e32a => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:36.828045 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c9248-3373-4014-a10e-79e4936dd332 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:36.841162 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:36.951729 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c9248-3373-4014-a10e-79e4936dd332 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:36.993501 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a12b0-e0f3-427f-9c9d-986642ca17c6 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:36.997435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928358be-a688-4d98-865e-2f34aeefcffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:36.998291 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928358be-a688-4d98-865e-2f34aeefcffb None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:36.998736 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928358be-a688-4d98-865e-2f34aeefcffb None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:36.999195 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928358be-a688-4d98-865e-2f34aeefcffb None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:37.000124 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928358be-a688-4d98-865e-2f34aeefcffb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:37.000585 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928358be-a688-4d98-865e-2f34aeefcffb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:37.002771 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928358be-a688-4d98-865e-2f34aeefcffb None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:37.015498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 66/131] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:28:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:37.016394 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928358be-a688-4d98-865e-2f34aeefcffb None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:37.039026 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc13137-a8a8-4f5d-8e32-5017501fd9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:37.041012 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc13137-a8a8-4f5d-8e32-5017501fd9f3 None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:37.041510 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc13137-a8a8-4f5d-8e32-5017501fd9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:37.042875 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc13137-a8a8-4f5d-8e32-5017501fd9f3 None admin] PATH_INFO: `/v3/projects/8f586cc781db4f2b96f13e85c4851e5a/users/a476d9b478cd40989e9e4e02fbc699e5/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:37.045497 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc13137-a8a8-4f5d-8e32-5017501fd9f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=8f586cc781db4f2b96f13e85c4851e5a, user_id=a476d9b478cd40989e9e4e02fbc699e5, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:37.047928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc13137-a8a8-4f5d-8e32-5017501fd9f3 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:37.067978 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 66/132] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Jun 15 14:28:36 2022] PUT /identity/v3/projects/8f586cc781db4f2b96f13e85c4851e5a/users/a476d9b478cd40989e9e4e02fbc699e5/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:28:37.068578 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc13137-a8a8-4f5d-8e32-5017501fd9f3 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:37.117177 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:37.131618 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:37.162014 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928358be-a688-4d98-865e-2f34aeefcffb None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:37.207068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce636c6-9646-4b0f-a4ff-c5d30cbd48a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:37.208329 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce636c6-9646-4b0f-a4ff-c5d30cbd48a6 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:37.208889 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce636c6-9646-4b0f-a4ff-c5d30cbd48a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:37.209377 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce636c6-9646-4b0f-a4ff-c5d30cbd48a6 None admin] PATH_INFO: `/v3/projects/3c26a382a0344c9580d9217b0f98e064` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:37.210953 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce636c6-9646-4b0f-a4ff-c5d30cbd48a6 None admin] RBAC: Authorizing `identity:get_project(project_id=3c26a382a0344c9580d9217b0f98e064)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:37.212900 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce636c6-9646-4b0f-a4ff-c5d30cbd48a6 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:37.215888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 67/133] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:28:37 2022] GET /identity/v3/projects/3c26a382a0344c9580d9217b0f98e064 => generated 312 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:37.224292 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce636c6-9646-4b0f-a4ff-c5d30cbd48a6 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:37.318849 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc13137-a8a8-4f5d-8e32-5017501fd9f3 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:37.381555 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce636c6-9646-4b0f-a4ff-c5d30cbd48a6 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:37.386573 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9a331b-61dc-4d55-8214-b71ddbff3298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:37.387752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a331b-61dc-4d55-8214-b71ddbff3298 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:37.388236 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a331b-61dc-4d55-8214-b71ddbff3298 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:37.388721 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a331b-61dc-4d55-8214-b71ddbff3298 None admin] PATH_INFO: `/v3/users/34d8aa4f47ae452592b3481bdb02e32a` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:37.390215 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9a331b-61dc-4d55-8214-b71ddbff3298 None admin] RBAC: Authorizing `identity:get_user(user_id=34d8aa4f47ae452592b3481bdb02e32a)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:37.396079 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9a331b-61dc-4d55-8214-b71ddbff3298 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:37.398135 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 67/134] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:37 2022] GET /identity/v3/users/34d8aa4f47ae452592b3481bdb02e32a => generated 275 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:37.404132 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9a331b-61dc-4d55-8214-b71ddbff3298 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:37.419263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7d4ac9-c0ac-479b-9049-9f06087680fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:37.421936 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d4ac9-c0ac-479b-9049-9f06087680fb None admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:37.421936 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d4ac9-c0ac-479b-9049-9f06087680fb None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:37.421936 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d4ac9-c0ac-479b-9049-9f06087680fb None admin] PATH_INFO: `/v3/projects/3c26a382a0344c9580d9217b0f98e064/users/34d8aa4f47ae452592b3481bdb02e32a/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:37.423506 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7d4ac9-c0ac-479b-9049-9f06087680fb None admin] RBAC: Authorizing `identity:create_grant(project_id=3c26a382a0344c9580d9217b0f98e064, user_id=34d8aa4f47ae452592b3481bdb02e32a, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:37.425577 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7d4ac9-c0ac-479b-9049-9f06087680fb None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:37.446204 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 68/135] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Jun 15 14:28:37 2022] PUT /identity/v3/projects/3c26a382a0344c9580d9217b0f98e064/users/34d8aa4f47ae452592b3481bdb02e32a/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:28:37.446856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:37.537781 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:37.551203 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9a331b-61dc-4d55-8214-b71ddbff3298 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:37.593460 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c1b08d-585e-4b82-8634-ab4e1e8c5167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:37.594359 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1b08d-585e-4b82-8634-ab4e1e8c5167 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:37.594645 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1b08d-585e-4b82-8634-ab4e1e8c5167 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:37.594923 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1b08d-585e-4b82-8634-ab4e1e8c5167 None admin] PATH_INFO: `/v3/projects/8f586cc781db4f2b96f13e85c4851e5a` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:37.596258 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c1b08d-585e-4b82-8634-ab4e1e8c5167 None admin] RBAC: Authorizing `identity:get_project(project_id=8f586cc781db4f2b96f13e85c4851e5a)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:37.597830 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c1b08d-585e-4b82-8634-ab4e1e8c5167 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:37.599843 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 68/136] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:28:37 2022] GET /identity/v3/projects/8f586cc781db4f2b96f13e85c4851e5a => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:37.605659 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7d4ac9-c0ac-479b-9049-9f06087680fb None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:37.758186 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7d4ac9-c0ac-479b-9049-9f06087680fb None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:37.808378 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e408e3f-379f-4757-b56d-62fcb1aa9231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:37.809291 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e408e3f-379f-4757-b56d-62fcb1aa9231 None admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:37.809568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e408e3f-379f-4757-b56d-62fcb1aa9231 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:37.809841 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e408e3f-379f-4757-b56d-62fcb1aa9231 None admin] PATH_INFO: `/v3/projects/8f586cc781db4f2b96f13e85c4851e5a/users/34d8aa4f47ae452592b3481bdb02e32a/roles/6b6353b49c6a41b1a157f8997fc52d9a` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:37.812327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e408e3f-379f-4757-b56d-62fcb1aa9231 None admin] RBAC: Authorizing `identity:create_grant(project_id=8f586cc781db4f2b96f13e85c4851e5a, user_id=34d8aa4f47ae452592b3481bdb02e32a, role_id=6b6353b49c6a41b1a157f8997fc52d9a)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:37.813826 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e408e3f-379f-4757-b56d-62fcb1aa9231 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:37.847996 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 69/137] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Jun 15 14:28:37 2022] PUT /identity/v3/projects/8f586cc781db4f2b96f13e85c4851e5a/users/34d8aa4f47ae452592b3481bdb02e32a/roles/6b6353b49c6a41b1a157f8997fc52d9a => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:28:37.893190 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132a2a4-c7e0-4d8c-bf9f-6addfb76bae0 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:37.893784 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132a2a4-c7e0-4d8c-bf9f-6addfb76bae0 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:37.893784 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132a2a4-c7e0-4d8c-bf9f-6addfb76bae0 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:37.895561 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 69/138] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:37.917248 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44456cf2-881b-4c02-b982-f3406250e0d0 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:37.917596 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44456cf2-881b-4c02-b982-f3406250e0d0 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:37.918040 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44456cf2-881b-4c02-b982-f3406250e0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:37.947110 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-44456cf2-881b-4c02-b982-f3406250e0d0 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:37.948495 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-44456cf2-881b-4c02-b982-f3406250e0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:38.040376 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:38.066204 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 70/139] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:38.098240 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36f56b-2609-45b8-b1c1-5f5bab035a76 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:38.098565 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36f56b-2609-45b8-b1c1-5f5bab035a76 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:38.098816 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36f56b-2609-45b8-b1c1-5f5bab035a76 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:38.100638 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 70/140] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:38.106696 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ccedf-059b-4d9e-9354-674b24c01269 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:38.107280 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ccedf-059b-4d9e-9354-674b24c01269 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:38.107617 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ccedf-059b-4d9e-9354-674b24c01269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:38.116712 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36f56b-2609-45b8-b1c1-5f5bab035a76 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:38.141258 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-958ccedf-059b-4d9e-9354-674b24c01269 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:38.142753 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-958ccedf-059b-4d9e-9354-674b24c01269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:38.260497 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36f56b-2609-45b8-b1c1-5f5bab035a76 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:38.274800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 71/141] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:38.306943 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfec6b7-7a2f-4239-946d-5455964da700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:38.308386 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfec6b7-7a2f-4239-946d-5455964da700 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:38.308679 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfec6b7-7a2f-4239-946d-5455964da700 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:38.308957 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfec6b7-7a2f-4239-946d-5455964da700 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:38.310497 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfec6b7-7a2f-4239-946d-5455964da700 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:38.312019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfec6b7-7a2f-4239-946d-5455964da700 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:38.313946 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 71/142] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:28:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:38.320472 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ccedf-059b-4d9e-9354-674b24c01269 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:38.324241 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfec6b7-7a2f-4239-946d-5455964da700 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:38.355647 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:38.551803 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ccedf-059b-4d9e-9354-674b24c01269 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:38.556984 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfec6b7-7a2f-4239-946d-5455964da700 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:38.590606 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:38.622451 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac195507-db86-4a16-89aa-89205f4b3206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:38.623766 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac195507-db86-4a16-89aa-89205f4b3206 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:38.624247 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac195507-db86-4a16-89aa-89205f4b3206 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:38.624749 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac195507-db86-4a16-89aa-89205f4b3206 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:38.635773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2622a433-bb6a-48b3-ad13-be4378e41b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:38.636978 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622a433-bb6a-48b3-ad13-be4378e41b0e None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:38.637256 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622a433-bb6a-48b3-ad13-be4378e41b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:38.637527 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622a433-bb6a-48b3-ad13-be4378e41b0e None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:38.638257 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2622a433-bb6a-48b3-ad13-be4378e41b0e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:38.639733 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2622a433-bb6a-48b3-ad13-be4378e41b0e None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:38.656572 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac195507-db86-4a16-89aa-89205f4b3206 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:38.668834 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-ac195507-db86-4a16-89aa-89205f4b3206 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:28:38.668834 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:38.668834 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:38.668834 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:38.668834 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:38.668834 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:38.668834 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:38.668834 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:38.668834 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:38.668834 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:38.668834 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:38.668834 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:38.668834 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:38.668834 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:38.668834 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:38.668834 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:38.669770 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:38.669770 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:38.669770 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:38.669770 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:38.669770 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:38.669770 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:38.669770 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:38.669770 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:38.669770 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:38.669770 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:38.669770 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:38.669770 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:38.669770 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:38.669770 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:38.669770 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:38.669770 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:38.670521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:38.670521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:38.670521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:38.670521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:38.670521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:38.670521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:38.670521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:38.670521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:38.670521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:38.670521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:38.670521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:28:38.670521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:28:38.670521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 72/143] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:28:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:28:38.676252 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac195507-db86-4a16-89aa-89205f4b3206 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:38.693223 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 72/144] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Jun 15 14:28:38 2022] POST /identity/v3/users => generated 283 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:28:38.846506 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac195507-db86-4a16-89aa-89205f4b3206 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:38.897140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db3a6a-18f7-49f6-a1f5-e2c20f015b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:38.897817 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db3a6a-18f7-49f6-a1f5-e2c20f015b49 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:38.898109 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db3a6a-18f7-49f6-a1f5-e2c20f015b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:38.898390 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db3a6a-18f7-49f6-a1f5-e2c20f015b49 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:38.899176 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db3a6a-18f7-49f6-a1f5-e2c20f015b49 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:38.899502 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db3a6a-18f7-49f6-a1f5-e2c20f015b49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:38.900903 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db3a6a-18f7-49f6-a1f5-e2c20f015b49 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:38.915308 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 73/145] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Jun 15 14:28:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:38.921030 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2622a433-bb6a-48b3-ad13-be4378e41b0e None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:39.043931 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:39.067028 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2622a433-bb6a-48b3-ad13-be4378e41b0e None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:39.106206 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae5eb3d-060c-47ff-88f9-0c1075c418fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:39.107474 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5eb3d-060c-47ff-88f9-0c1075c418fa None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:39.107958 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5eb3d-060c-47ff-88f9-0c1075c418fa None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:39.108457 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5eb3d-060c-47ff-88f9-0c1075c418fa None admin] PATH_INFO: `/v3/users/4e1d412ac8d84b70a22ec80f41fded6c` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:39.109920 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae5eb3d-060c-47ff-88f9-0c1075c418fa None admin] RBAC: Authorizing `identity:get_user(user_id=4e1d412ac8d84b70a22ec80f41fded6c)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:39.111672 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae5eb3d-060c-47ff-88f9-0c1075c418fa None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:39.113947 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 73/146] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:38 2022] GET /identity/v3/users/4e1d412ac8d84b70a22ec80f41fded6c => generated 289 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:39.119176 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db3a6a-18f7-49f6-a1f5-e2c20f015b49 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:39.180156 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:39.246480 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db3a6a-18f7-49f6-a1f5-e2c20f015b49 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:39.286439 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7974534c-1c15-49d6-ad06-08176fdfc6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:39.287346 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974534c-1c15-49d6-ad06-08176fdfc6bb None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:39.287633 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974534c-1c15-49d6-ad06-08176fdfc6bb None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:39.287912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974534c-1c15-49d6-ad06-08176fdfc6bb None admin] PATH_INFO: `/v3/projects/8f586cc781db4f2b96f13e85c4851e5a` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:39.291712 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7974534c-1c15-49d6-ad06-08176fdfc6bb None admin] RBAC: Authorizing `identity:get_project(project_id=8f586cc781db4f2b96f13e85c4851e5a)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:39.293758 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7974534c-1c15-49d6-ad06-08176fdfc6bb None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:39.295508 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 74/147] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:28:39 2022] GET /identity/v3/projects/8f586cc781db4f2b96f13e85c4851e5a => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:39.301198 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae5eb3d-060c-47ff-88f9-0c1075c418fa None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:39.423558 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae5eb3d-060c-47ff-88f9-0c1075c418fa None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:39.466821 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e4c91-c57f-46c8-97f4-5e8b2359d016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:39.467794 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e4c91-c57f-46c8-97f4-5e8b2359d016 None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:39.468083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e4c91-c57f-46c8-97f4-5e8b2359d016 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:39.468358 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e4c91-c57f-46c8-97f4-5e8b2359d016 None admin] PATH_INFO: `/v3/projects/8f586cc781db4f2b96f13e85c4851e5a/users/4e1d412ac8d84b70a22ec80f41fded6c/roles/30e8990a9ea24c4398bbe057f41400bb` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:39.470354 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61e4c91-c57f-46c8-97f4-5e8b2359d016 None admin] RBAC: Authorizing `identity:create_grant(project_id=8f586cc781db4f2b96f13e85c4851e5a, user_id=4e1d412ac8d84b70a22ec80f41fded6c, role_id=30e8990a9ea24c4398bbe057f41400bb)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:39.471859 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61e4c91-c57f-46c8-97f4-5e8b2359d016 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:39.495887 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 74/148] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Jun 15 14:28:39 2022] PUT /identity/v3/projects/8f586cc781db4f2b96f13e85c4851e5a/users/4e1d412ac8d84b70a22ec80f41fded6c/roles/30e8990a9ea24c4398bbe057f41400bb => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:28:40.217595 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab271241-4034-4593-8d8e-8851723c24e3 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:40.218102 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab271241-4034-4593-8d8e-8851723c24e3 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:40.218176 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab271241-4034-4593-8d8e-8851723c24e3 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:40.219609 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 75/149] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:40.225137 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a03c3b-9e40-4f9e-b05a-f4a02c0e56b6 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:40.225578 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a03c3b-9e40-4f9e-b05a-f4a02c0e56b6 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:40.225758 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a03c3b-9e40-4f9e-b05a-f4a02c0e56b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:40.245479 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-e5a03c3b-9e40-4f9e-b05a-f4a02c0e56b6 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:40.246665 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e5a03c3b-9e40-4f9e-b05a-f4a02c0e56b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:40.248777 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:40.332848 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 75/150] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:40.373647 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab271241-4034-4593-8d8e-8851723c24e3 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:40.508159 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab271241-4034-4593-8d8e-8851723c24e3 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:40.518632 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:40.554743 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf5b20-580f-4ad8-808d-17b156277c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:40.555583 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf5b20-580f-4ad8-808d-17b156277c1c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:40.555874 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf5b20-580f-4ad8-808d-17b156277c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:40.556154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf5b20-580f-4ad8-808d-17b156277c1c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:40.572665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf5b20-580f-4ad8-808d-17b156277c1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:40.581206 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-1ccf5b20-580f-4ad8-808d-17b156277c1c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:28:40.581206 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:40.581206 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:40.581206 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:40.581206 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:40.581206 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:40.581206 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:40.581206 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:40.581206 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:40.581206 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:40.581206 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:40.581206 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:40.581206 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:40.581206 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:40.581206 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:40.581206 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:40.582037 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:40.582037 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:40.582037 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:40.582037 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:40.582037 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:40.582037 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:40.582037 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:40.582037 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:40.582037 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:40.582037 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:40.582037 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:40.582037 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:40.582037 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:40.582037 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:40.582037 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:40.582037 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:40.582834 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:40.582834 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:40.582834 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:40.582834 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:40.582834 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:40.582834 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:40.582834 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:40.582834 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:40.582834 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:40.582834 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:40.582834 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:28:40.582834 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:28:40.582834 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 76/151] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:28:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:28:40.590123 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a03c3b-9e40-4f9e-b05a-f4a02c0e56b6 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:40.597065 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb6166-4f62-488c-a48f-b0376f0df3c1 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:40.597361 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb6166-4f62-488c-a48f-b0376f0df3c1 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:40.597645 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb6166-4f62-488c-a48f-b0376f0df3c1 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:40.599064 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 77/152] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:28:40.605227 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789c267-b7fc-4d75-a427-1b9420ac403f None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:40.605530 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789c267-b7fc-4d75-a427-1b9420ac403f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:40.605820 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789c267-b7fc-4d75-a427-1b9420ac403f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:40.628569 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-a789c267-b7fc-4d75-a427-1b9420ac403f None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:40.629789 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-a789c267-b7fc-4d75-a427-1b9420ac403f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:40.713664 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 78/153] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:40.742625 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789c267-b7fc-4d75-a427-1b9420ac403f None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:40.788208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a03c3b-9e40-4f9e-b05a-f4a02c0e56b6 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:40.859879 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc13820a-c6f7-46b1-9b9d-a35e35456274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:40.860616 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13820a-c6f7-46b1-9b9d-a35e35456274 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:40.860917 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13820a-c6f7-46b1-9b9d-a35e35456274 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:40.861631 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13820a-c6f7-46b1-9b9d-a35e35456274 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:40.862414 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc13820a-c6f7-46b1-9b9d-a35e35456274 None admin] RBAC: Adding query filter params (name=member) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:40.862728 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc13820a-c6f7-46b1-9b9d-a35e35456274 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:40.866319 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc13820a-c6f7-46b1-9b9d-a35e35456274 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:40.876435 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:40.882763 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 76/154] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Jun 15 14:28:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:40.889292 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc13820a-c6f7-46b1-9b9d-a35e35456274 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:40.904970 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:40.912832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789c267-b7fc-4d75-a427-1b9420ac403f None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:40.962825 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b147877-5d3d-45f1-a80f-710dda039857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:40.963667 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b147877-5d3d-45f1-a80f-710dda039857 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:40.964489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b147877-5d3d-45f1-a80f-710dda039857 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:40.964771 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b147877-5d3d-45f1-a80f-710dda039857 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:40.975707 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b147877-5d3d-45f1-a80f-710dda039857 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:40.984244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-6b147877-5d3d-45f1-a80f-710dda039857 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:28:40.984244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:40.984244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:40.984244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:40.984244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:40.984244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:40.984244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:40.984244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:40.984244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:40.984244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:40.984244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:40.984244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:40.984244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:40.984244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:40.984244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:40.984244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:40.985064 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:40.985064 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:40.985064 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:40.985064 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:40.985064 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:40.985064 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:40.985064 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:40.985064 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:40.985064 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:40.985064 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:40.985064 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:40.985064 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:40.985064 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:40.985064 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:40.985064 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:40.985064 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:40.985953 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:40.985953 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:40.985953 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:40.985953 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:40.985953 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:40.985953 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:40.985953 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:40.985953 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:40.985953 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:40.985953 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:40.985953 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:28:40.985953 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:28:40.985953 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 79/155] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:28:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:28:40.987357 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3dea3-7786-4f87-879b-e5da29c3f2c3 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:40.987634 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3dea3-7786-4f87-879b-e5da29c3f2c3 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:40.988132 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3dea3-7786-4f87-879b-e5da29c3f2c3 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:40.989228 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 80/156] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:40.990440 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c190c-4b38-4a54-a2c7-58272e891b66 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:40.990790 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c190c-4b38-4a54-a2c7-58272e891b66 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:40.991092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c190c-4b38-4a54-a2c7-58272e891b66 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:40.992808 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 81/157] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:40.995601 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ada9f-6d3d-41d7-967b-23f906a1ee29 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:40.995904 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ada9f-6d3d-41d7-967b-23f906a1ee29 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:40.996239 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ada9f-6d3d-41d7-967b-23f906a1ee29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:41.013848 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-7c9ada9f-6d3d-41d7-967b-23f906a1ee29 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:41.015297 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-7c9ada9f-6d3d-41d7-967b-23f906a1ee29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:41.054075 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc13820a-c6f7-46b1-9b9d-a35e35456274 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:41.087533 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 82/158] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:41.088544 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ada9f-6d3d-41d7-967b-23f906a1ee29 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:41.114442 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7454584-41e4-4975-8812-fc56bf012547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:41.115393 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7454584-41e4-4975-8812-fc56bf012547 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:41.115724 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7454584-41e4-4975-8812-fc56bf012547 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:41.116006 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7454584-41e4-4975-8812-fc56bf012547 None admin] PATH_INFO: `/v3/users/34d8aa4f47ae452592b3481bdb02e32a` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:41.117243 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7454584-41e4-4975-8812-fc56bf012547 None admin] RBAC: Authorizing `identity:get_user(user_id=34d8aa4f47ae452592b3481bdb02e32a)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:41.118712 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7454584-41e4-4975-8812-fc56bf012547 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:41.120731 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 77/159] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:40 2022] GET /identity/v3/users/34d8aa4f47ae452592b3481bdb02e32a => generated 275 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:41.121936 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef93ec8-cc36-4d3a-8374-bcd535ed8e37 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:41.122224 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef93ec8-cc36-4d3a-8374-bcd535ed8e37 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:41.122500 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef93ec8-cc36-4d3a-8374-bcd535ed8e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:41.157500 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-1ef93ec8-cc36-4d3a-8374-bcd535ed8e37 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:41.158711 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1ef93ec8-cc36-4d3a-8374-bcd535ed8e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:41.186814 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:41.199837 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ada9f-6d3d-41d7-967b-23f906a1ee29 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:41.229605 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:41.239450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 78/160] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:41.240544 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef93ec8-cc36-4d3a-8374-bcd535ed8e37 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:41.242102 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a492cc-0a42-4816-9a9d-c15553ee3c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:41.242780 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a492cc-0a42-4816-9a9d-c15553ee3c54 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:41.243192 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a492cc-0a42-4816-9a9d-c15553ee3c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:41.245451 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a492cc-0a42-4816-9a9d-c15553ee3c54 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:41.246291 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a492cc-0a42-4816-9a9d-c15553ee3c54 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:41.246589 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a492cc-0a42-4816-9a9d-c15553ee3c54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:41.248913 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a492cc-0a42-4816-9a9d-c15553ee3c54 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:41.268585 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 83/161] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:28:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:41.269516 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a492cc-0a42-4816-9a9d-c15553ee3c54 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:41.368476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef93ec8-cc36-4d3a-8374-bcd535ed8e37 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:41.373901 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a492cc-0a42-4816-9a9d-c15553ee3c54 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:41.407228 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c63d2-1feb-4f46-91a2-0637d4ef0f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:41.408054 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c63d2-1feb-4f46-91a2-0637d4ef0f3e None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:41.408344 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c63d2-1feb-4f46-91a2-0637d4ef0f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:41.408624 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c63d2-1feb-4f46-91a2-0637d4ef0f3e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:41.409911 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e5cee-c73d-40b5-931c-8b95bac8d36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:41.410575 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e5cee-c73d-40b5-931c-8b95bac8d36e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:41.410861 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e5cee-c73d-40b5-931c-8b95bac8d36e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:41.411178 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e5cee-c73d-40b5-931c-8b95bac8d36e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:41.420147 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c63d2-1feb-4f46-91a2-0637d4ef0f3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:41.428819 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-591c63d2-1feb-4f46-91a2-0637d4ef0f3e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:28:41.428819 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:41.428819 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:41.428819 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:41.428819 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:41.428819 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:41.428819 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:41.428819 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:41.428819 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:41.428819 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:41.428819 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:41.428819 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:41.428819 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:41.428819 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:41.428819 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:41.428819 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:41.430633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:41.430633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:41.430633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:41.430633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:41.430633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:41.430633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:41.430633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:41.430633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:41.430633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:41.430633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:41.430633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:41.430633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:41.430633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:41.430633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:41.430633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:41.430633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:41.431416 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:41.431416 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:41.431416 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:41.431416 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:41.431416 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:41.431416 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:41.431416 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:41.431416 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:41.431416 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:41.431416 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:41.431416 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:28:41.431416 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:28:41.431416 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 79/162] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Jun 15 14:28:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:28:41.432041 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:41.432108 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c63d2-1feb-4f46-91a2-0637d4ef0f3e None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:41.485151 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570e5cee-c73d-40b5-931c-8b95bac8d36e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:41.486647 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570e5cee-c73d-40b5-931c-8b95bac8d36e None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:41.535016 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:41.589456 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 84/163] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:28:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:28:41.594750 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e5cee-c73d-40b5-931c-8b95bac8d36e None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:41.607656 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c63d2-1feb-4f46-91a2-0637d4ef0f3e None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:41.647586 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776bbc72-bd60-4293-9ee8-5f534afbb28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:41.648309 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bbc72-bd60-4293-9ee8-5f534afbb28c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:41.648598 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bbc72-bd60-4293-9ee8-5f534afbb28c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:41.648879 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bbc72-bd60-4293-9ee8-5f534afbb28c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:41.659652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776bbc72-bd60-4293-9ee8-5f534afbb28c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:41.667563 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-776bbc72-bd60-4293-9ee8-5f534afbb28c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:28:41.667563 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:41.667563 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:41.667563 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:41.667563 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:41.667563 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:41.667563 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:41.667563 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:41.667563 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:41.667563 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:41.667563 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:41.667563 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:41.667563 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:41.667563 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:41.667563 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:41.667563 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:41.668633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:41.668633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:41.668633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:41.668633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:41.668633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:41.668633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:41.668633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:41.668633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:41.668633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:41.668633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:41.668633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:41.668633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:41.668633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:41.668633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:41.668633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:41.668633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:41.669384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:41.669384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:41.669384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:41.669384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:41.669384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:41.669384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:41.669384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:41.669384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:41.669384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:41.669384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:41.669384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:28:41.669384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:28:41.669384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 80/164] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:28:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:28:41.670015 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776bbc72-bd60-4293-9ee8-5f534afbb28c None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:41.675353 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:41.744070 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e5cee-c73d-40b5-931c-8b95bac8d36e None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:41.816278 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db532431-aced-4643-840e-5b4480ecedf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:41.817167 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db532431-aced-4643-840e-5b4480ecedf8 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:41.817457 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db532431-aced-4643-840e-5b4480ecedf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:41.817724 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db532431-aced-4643-840e-5b4480ecedf8 None admin] PATH_INFO: `/v3/users/a476d9b478cd40989e9e4e02fbc699e5` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:41.823787 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db532431-aced-4643-840e-5b4480ecedf8 None admin] RBAC: Authorizing `identity:get_user(user_id=a476d9b478cd40989e9e4e02fbc699e5)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:41.825061 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776bbc72-bd60-4293-9ee8-5f534afbb28c None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:41.825323 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db532431-aced-4643-840e-5b4480ecedf8 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:41.827234 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 85/165] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:41 2022] GET /identity/v3/users/a476d9b478cd40989e9e4e02fbc699e5 => generated 247 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:41.827922 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db532431-aced-4643-840e-5b4480ecedf8 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:41.860122 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:41.900742 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c5487-a222-4f0b-a345-c7a610eb9f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:41.902313 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c5487-a222-4f0b-a345-c7a610eb9f49 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:41.902893 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c5487-a222-4f0b-a345-c7a610eb9f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:41.903388 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c5487-a222-4f0b-a345-c7a610eb9f49 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:41.904454 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1c5487-a222-4f0b-a345-c7a610eb9f49 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:41.904959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1c5487-a222-4f0b-a345-c7a610eb9f49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:41.906670 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1c5487-a222-4f0b-a345-c7a610eb9f49 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:41.926501 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 81/166] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:28:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:41.927759 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c5487-a222-4f0b-a345-c7a610eb9f49 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:41.956229 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db532431-aced-4643-840e-5b4480ecedf8 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:41.979379 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:41.998692 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc608f9-0154-4565-b654-05d3306196d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:41.999643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc608f9-0154-4565-b654-05d3306196d8 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:41.999922 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc608f9-0154-4565-b654-05d3306196d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:42.000194 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc608f9-0154-4565-b654-05d3306196d8 None admin] PATH_INFO: `/v3/projects/8f586cc781db4f2b96f13e85c4851e5a` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:42.001430 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc608f9-0154-4565-b654-05d3306196d8 None admin] RBAC: Authorizing `identity:get_project(project_id=8f586cc781db4f2b96f13e85c4851e5a)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:42.002932 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc608f9-0154-4565-b654-05d3306196d8 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:42.004742 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 86/167] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:28:41 2022] GET /identity/v3/projects/8f586cc781db4f2b96f13e85c4851e5a => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:42.006708 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc608f9-0154-4565-b654-05d3306196d8 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:42.097776 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c5487-a222-4f0b-a345-c7a610eb9f49 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:42.117342 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc608f9-0154-4565-b654-05d3306196d8 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:42.132522 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6307f921-e42f-43ed-8eb0-844d1a90d9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:42.133203 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307f921-e42f-43ed-8eb0-844d1a90d9e1 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:42.133489 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307f921-e42f-43ed-8eb0-844d1a90d9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:42.133768 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307f921-e42f-43ed-8eb0-844d1a90d9e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:42.134532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6307f921-e42f-43ed-8eb0-844d1a90d9e1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:42.134841 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6307f921-e42f-43ed-8eb0-844d1a90d9e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:42.136830 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6307f921-e42f-43ed-8eb0-844d1a90d9e1 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:42.145756 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 82/168] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Jun 15 14:28:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:42.146392 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6307f921-e42f-43ed-8eb0-844d1a90d9e1 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:42.159743 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96056d7-788d-48eb-94b6-4ed2cfc466a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:42.160328 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96056d7-788d-48eb-94b6-4ed2cfc466a0 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:42.160613 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96056d7-788d-48eb-94b6-4ed2cfc466a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:42.160942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96056d7-788d-48eb-94b6-4ed2cfc466a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:42.198186 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:42.239418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:42.253078 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96056d7-788d-48eb-94b6-4ed2cfc466a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:42.254608 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96056d7-788d-48eb-94b6-4ed2cfc466a0 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:42.352909 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6307f921-e42f-43ed-8eb0-844d1a90d9e1 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:42.362472 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 87/169] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:28:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:28:42.363457 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96056d7-788d-48eb-94b6-4ed2cfc466a0 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:42.418952 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7282abd-61af-4958-a097-ec3e7c273ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:42.419783 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7282abd-61af-4958-a097-ec3e7c273ddb None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:42.420075 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7282abd-61af-4958-a097-ec3e7c273ddb None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:42.420356 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7282abd-61af-4958-a097-ec3e7c273ddb None admin] PATH_INFO: `/v3/users/34d8aa4f47ae452592b3481bdb02e32a` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:42.421579 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7282abd-61af-4958-a097-ec3e7c273ddb None admin] RBAC: Authorizing `identity:get_user(user_id=34d8aa4f47ae452592b3481bdb02e32a)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:42.423078 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7282abd-61af-4958-a097-ec3e7c273ddb None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:42.425043 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 83/170] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:42 2022] GET /identity/v3/users/34d8aa4f47ae452592b3481bdb02e32a => generated 275 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:42.425983 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7282abd-61af-4958-a097-ec3e7c273ddb None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:42.463369 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:42.476941 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96056d7-788d-48eb-94b6-4ed2cfc466a0 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:42.533012 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7282abd-61af-4958-a097-ec3e7c273ddb None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:42.540181 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db6bfb4-3e94-4b57-a556-ccbafcfec4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:42.540966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6bfb4-3e94-4b57-a556-ccbafcfec4ef None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:42.541258 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6bfb4-3e94-4b57-a556-ccbafcfec4ef None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:42.541538 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6bfb4-3e94-4b57-a556-ccbafcfec4ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:42.542412 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db6bfb4-3e94-4b57-a556-ccbafcfec4ef None admin] RBAC: Adding query filter params (role.id=ce0de9b9c9054826b569db895942ad26, scope.project.id=8f586cc781db4f2b96f13e85c4851e5a, user.id=34d8aa4f47ae452592b3481bdb02e32a) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:42.542831 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db6bfb4-3e94-4b57-a556-ccbafcfec4ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:42.544440 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:42.544925 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db6bfb4-3e94-4b57-a556-ccbafcfec4ef None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:42.555586 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 88/171] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:28:42 2022] GET /identity/v3/role_assignments?user.id=34d8aa4f47ae452592b3481bdb02e32a&scope.project.id=8f586cc781db4f2b96f13e85c4851e5a&role.id=ce0de9b9c9054826b569db895942ad26 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:42.560139 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db6bfb4-3e94-4b57-a556-ccbafcfec4ef None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:42.581614 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c768387-8c42-43eb-b98c-7990e92c59db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:42.582515 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c768387-8c42-43eb-b98c-7990e92c59db None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:42.582803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c768387-8c42-43eb-b98c-7990e92c59db None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:42.583106 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c768387-8c42-43eb-b98c-7990e92c59db None admin] PATH_INFO: `/v3/users/34d8aa4f47ae452592b3481bdb02e32a` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:42.584443 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c768387-8c42-43eb-b98c-7990e92c59db None admin] RBAC: Authorizing `identity:get_user(user_id=34d8aa4f47ae452592b3481bdb02e32a)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:42.585964 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c768387-8c42-43eb-b98c-7990e92c59db None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:42.595220 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 84/172] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:42 2022] GET /identity/v3/users/34d8aa4f47ae452592b3481bdb02e32a => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:42.599069 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c768387-8c42-43eb-b98c-7990e92c59db None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:42.808349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db6bfb4-3e94-4b57-a556-ccbafcfec4ef None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:42.832650 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c768387-8c42-43eb-b98c-7990e92c59db None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:42.876666 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f17a871-2730-43e6-b888-3604838cc20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:42.877934 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f17a871-2730-43e6-b888-3604838cc20f None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:42.879507 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f17a871-2730-43e6-b888-3604838cc20f None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:42.880643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f17a871-2730-43e6-b888-3604838cc20f None admin] PATH_INFO: `/v3/projects/8f586cc781db4f2b96f13e85c4851e5a` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:42.882740 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f17a871-2730-43e6-b888-3604838cc20f None admin] RBAC: Authorizing `identity:get_project(project_id=8f586cc781db4f2b96f13e85c4851e5a)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:42.884557 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f17a871-2730-43e6-b888-3604838cc20f None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:42.887556 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 89/173] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:28:42 2022] GET /identity/v3/projects/8f586cc781db4f2b96f13e85c4851e5a => generated 298 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:42.889199 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f17a871-2730-43e6-b888-3604838cc20f None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:42.899748 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0e7ac-f557-4ccf-abe1-1b3b47d7dc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:42.900627 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0e7ac-f557-4ccf-abe1-1b3b47d7dc0d None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:42.902124 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0e7ac-f557-4ccf-abe1-1b3b47d7dc0d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:42.902553 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0e7ac-f557-4ccf-abe1-1b3b47d7dc0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:42.939384 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:42.987432 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:43.026141 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd0e7ac-f557-4ccf-abe1-1b3b47d7dc0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:43.028200 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd0e7ac-f557-4ccf-abe1-1b3b47d7dc0d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:43.078969 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f17a871-2730-43e6-b888-3604838cc20f None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:43.139702 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c501600-8253-4a3b-ab83-7975b30ec496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:43.140361 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c501600-8253-4a3b-ab83-7975b30ec496 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:43.140652 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c501600-8253-4a3b-ab83-7975b30ec496 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:43.140926 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c501600-8253-4a3b-ab83-7975b30ec496 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:43.217783 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 85/174] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:28:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 622 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:28:43.218458 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0e7ac-f557-4ccf-abe1-1b3b47d7dc0d None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:43.291795 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c501600-8253-4a3b-ab83-7975b30ec496 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:43.293801 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c501600-8253-4a3b-ab83-7975b30ec496 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:43.342019 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:43.440869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:43.452714 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 90/175] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:28:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:28:43.454616 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c501600-8253-4a3b-ab83-7975b30ec496 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:43.479426 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0e7ac-f557-4ccf-abe1-1b3b47d7dc0d None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:43.522488 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b7f062-eb05-4cba-8c62-0848657ec770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:43.523457 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7f062-eb05-4cba-8c62-0848657ec770 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:43.523880 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7f062-eb05-4cba-8c62-0848657ec770 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:43.524289 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7f062-eb05-4cba-8c62-0848657ec770 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:43.525262 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b7f062-eb05-4cba-8c62-0848657ec770 None admin] RBAC: Adding query filter params (role.id=ed1cfa2fcf724a4d84f5acea100930c7, scope.project.id=8f586cc781db4f2b96f13e85c4851e5a, user.id=a476d9b478cd40989e9e4e02fbc699e5) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:43.528544 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b7f062-eb05-4cba-8c62-0848657ec770 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:43.530387 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b7f062-eb05-4cba-8c62-0848657ec770 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:43.553071 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 86/176] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:28:43 2022] GET /identity/v3/role_assignments?user.id=a476d9b478cd40989e9e4e02fbc699e5&scope.project.id=8f586cc781db4f2b96f13e85c4851e5a&role.id=ed1cfa2fcf724a4d84f5acea100930c7 => generated 608 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:43.554259 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b7f062-eb05-4cba-8c62-0848657ec770 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:43.581203 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c501600-8253-4a3b-ab83-7975b30ec496 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:43.623402 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:43.647266 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c449c-1584-4eed-981a-e839e4ff100a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:43.648142 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c449c-1584-4eed-981a-e839e4ff100a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:43.648424 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c449c-1584-4eed-981a-e839e4ff100a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:43.648696 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c449c-1584-4eed-981a-e839e4ff100a None admin] PATH_INFO: `/v3/projects/8f586cc781db4f2b96f13e85c4851e5a` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:43.650003 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c449c-1584-4eed-981a-e839e4ff100a None admin] RBAC: Authorizing `identity:get_project(project_id=8f586cc781db4f2b96f13e85c4851e5a)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:43.652070 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c449c-1584-4eed-981a-e839e4ff100a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:43.653891 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 91/177] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:28:43 2022] GET /identity/v3/projects/8f586cc781db4f2b96f13e85c4851e5a => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:43.655200 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3e57f-b7bc-4db4-9953-e4c820ecd594 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:43.655488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3e57f-b7bc-4db4-9953-e4c820ecd594 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:43.655771 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3e57f-b7bc-4db4-9953-e4c820ecd594 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:43.657013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 92/178] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:43.669463 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f3e57f-b7bc-4db4-9953-e4c820ecd594 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:43.835431 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b7f062-eb05-4cba-8c62-0848657ec770 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:43.876742 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f3e57f-b7bc-4db4-9953-e4c820ecd594 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:43.901641 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ecbf9-62c2-48c3-bc28-e1947315073c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:43.902530 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ecbf9-62c2-48c3-bc28-e1947315073c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:43.902820 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ecbf9-62c2-48c3-bc28-e1947315073c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:43.903116 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ecbf9-62c2-48c3-bc28-e1947315073c None admin] PATH_INFO: `/v3/projects/3c26a382a0344c9580d9217b0f98e064` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:43.904647 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1ecbf9-62c2-48c3-bc28-e1947315073c None admin] RBAC: Authorizing `identity:get_project(project_id=3c26a382a0344c9580d9217b0f98e064)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:43.907372 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1ecbf9-62c2-48c3-bc28-e1947315073c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:43.910308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 87/179] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:28:43 2022] GET /identity/v3/projects/3c26a382a0344c9580d9217b0f98e064 => generated 312 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:43.911972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94026f1-91cf-434d-a739-ada2fe8407b5 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:43.912423 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94026f1-91cf-434d-a739-ada2fe8407b5 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:43.912856 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94026f1-91cf-434d-a739-ada2fe8407b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:43.929747 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95228587-3d43-48b4-91dc-4da9310ebbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:43.930826 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95228587-3d43-48b4-91dc-4da9310ebbbc None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:43.931341 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95228587-3d43-48b4-91dc-4da9310ebbbc None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:43.931786 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95228587-3d43-48b4-91dc-4da9310ebbbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:43.933350 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95228587-3d43-48b4-91dc-4da9310ebbbc None admin] RBAC: Adding query filter params (role.id=6b6353b49c6a41b1a157f8997fc52d9a, scope.project.id=8f586cc781db4f2b96f13e85c4851e5a, user.id=34d8aa4f47ae452592b3481bdb02e32a) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:43.933815 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95228587-3d43-48b4-91dc-4da9310ebbbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:43.935228 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-d94026f1-91cf-434d-a739-ada2fe8407b5 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:43.935618 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95228587-3d43-48b4-91dc-4da9310ebbbc None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:43.936828 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d94026f1-91cf-434d-a739-ada2fe8407b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:43.941566 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:43.945567 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 93/180] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:28:43 2022] GET /identity/v3/role_assignments?user.id=34d8aa4f47ae452592b3481bdb02e32a&scope.project.id=8f586cc781db4f2b96f13e85c4851e5a&role.id=6b6353b49c6a41b1a157f8997fc52d9a => generated 608 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:43.947297 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ca670-8e75-43d5-89b7-d7663f31dba0 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:43.947726 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ca670-8e75-43d5-89b7-d7663f31dba0 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:43.948144 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ca670-8e75-43d5-89b7-d7663f31dba0 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:43.950250 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 94/181] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:43.951366 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ca670-8e75-43d5-89b7-d7663f31dba0 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:44.039914 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:44.063968 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 88/182] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:44.065984 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a5bdb8-8355-4823-ba3d-8492604a8ea2 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:44.067013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a5bdb8-8355-4823-ba3d-8492604a8ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:44.067447 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a5bdb8-8355-4823-ba3d-8492604a8ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:44.091217 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-a8a5bdb8-8355-4823-ba3d-8492604a8ea2 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:44.092363 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a8a5bdb8-8355-4823-ba3d-8492604a8ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:44.110747 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ca670-8e75-43d5-89b7-d7663f31dba0 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:44.159756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d7d2da-3d63-459d-9fbc-b8400073b9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:44.160429 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7d2da-3d63-459d-9fbc-b8400073b9ff None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:44.160713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7d2da-3d63-459d-9fbc-b8400073b9ff None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:44.160995 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7d2da-3d63-459d-9fbc-b8400073b9ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:44.162639 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d7d2da-3d63-459d-9fbc-b8400073b9ff None admin] RBAC: Adding query filter params (role.id=ce0de9b9c9054826b569db895942ad26, scope.project.id=3c26a382a0344c9580d9217b0f98e064, user.id=34d8aa4f47ae452592b3481bdb02e32a) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:44.162639 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d7d2da-3d63-459d-9fbc-b8400073b9ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:44.163649 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d7d2da-3d63-459d-9fbc-b8400073b9ff None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:44.169591 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 89/183] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:44.170916 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67884d1f-6415-40a7-aec8-0773c74ba260 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:44.171108 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 95/184] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:28:43 2022] GET /identity/v3/role_assignments?user.id=34d8aa4f47ae452592b3481bdb02e32a&scope.project.id=3c26a382a0344c9580d9217b0f98e064&role.id=ce0de9b9c9054826b569db895942ad26 => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:44.171287 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67884d1f-6415-40a7-aec8-0773c74ba260 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:44.171530 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67884d1f-6415-40a7-aec8-0773c74ba260 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:44.172354 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3d89a-fb87-4017-b9ba-a66d8e8b302c None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:44.172645 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3d89a-fb87-4017-b9ba-a66d8e8b302c None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:44.172801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 90/185] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:44.173495 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67884d1f-6415-40a7-aec8-0773c74ba260 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:44.183650 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3d89a-fb87-4017-b9ba-a66d8e8b302c None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:44.185372 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 96/186] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:44 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:44.193130 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462167d-7cf8-4761-9a4d-67ceb2bed456 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:44.193440 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462167d-7cf8-4761-9a4d-67ceb2bed456 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:44.193731 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462167d-7cf8-4761-9a4d-67ceb2bed456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:44.214165 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-c462167d-7cf8-4761-9a4d-67ceb2bed456 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:44.215423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c462167d-7cf8-4761-9a4d-67ceb2bed456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:44.276521 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:44.308451 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 97/187] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:44.310629 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac076e-df5a-4784-8c93-5cd3ac26b71d None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:44.310917 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac076e-df5a-4784-8c93-5cd3ac26b71d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:44.311228 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac076e-df5a-4784-8c93-5cd3ac26b71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:44.316807 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67884d1f-6415-40a7-aec8-0773c74ba260 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:44.336459 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-b6ac076e-df5a-4784-8c93-5cd3ac26b71d None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:44.338334 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b6ac076e-df5a-4784-8c93-5cd3ac26b71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:44.359931 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a782a1-a171-4ef4-a1ca-d1d7dd255f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:44.360662 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a782a1-a171-4ef4-a1ca-d1d7dd255f29 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:44.360949 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a782a1-a171-4ef4-a1ca-d1d7dd255f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:44.361234 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a782a1-a171-4ef4-a1ca-d1d7dd255f29 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:44.373411 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a782a1-a171-4ef4-a1ca-d1d7dd255f29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:44.381272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-42a782a1-a171-4ef4-a1ca-d1d7dd255f29 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:28:44.381272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:44.381272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:44.381272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:44.381272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:44.381272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:44.381272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:44.381272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:44.381272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:44.381272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:44.381272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:44.381272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:44.381272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:44.381272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:44.381272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:44.381272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:44.382371 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:44.382371 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:44.382371 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:44.382371 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:44.382371 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:44.382371 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:44.382371 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:44.382371 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:44.382371 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:44.382371 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:44.382371 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:44.382371 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:44.382371 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:44.382371 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:44.382371 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:44.382371 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:44.383290 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:44.383290 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:44.383290 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:44.383290 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:44.383290 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:44.383290 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:44.383290 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:44.383290 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:44.383290 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:44.383290 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:44.383290 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:28:44.383290 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:28:44.383290 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 91/188] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:28:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:28:44.384646 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:44.385671 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a782a1-a171-4ef4-a1ca-d1d7dd255f29 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:44.420237 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 98/189] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:44.431256 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac076e-df5a-4784-8c93-5cd3ac26b71d None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:44.489407 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a782a1-a171-4ef4-a1ca-d1d7dd255f29 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:44.517390 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:44.523575 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d7786-834a-4d2b-9bdc-379b6622b845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:44.524370 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d7786-834a-4d2b-9bdc-379b6622b845 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:44.524656 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d7786-834a-4d2b-9bdc-379b6622b845 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:44.524931 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d7786-834a-4d2b-9bdc-379b6622b845 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:44.527183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac076e-df5a-4784-8c93-5cd3ac26b71d None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:44.535539 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49d7786-834a-4d2b-9bdc-379b6622b845 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:44.543763 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-c49d7786-834a-4d2b-9bdc-379b6622b845 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:28:44.543763 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:44.543763 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:44.543763 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:44.543763 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:44.543763 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:44.543763 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:44.543763 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:44.543763 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:44.543763 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:44.543763 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:44.543763 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:44.543763 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:44.543763 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:44.543763 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:44.543763 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:44.544628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:44.544628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:44.544628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:44.544628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:44.544628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:44.544628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:44.544628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:44.544628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:44.544628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:44.544628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:44.544628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:44.544628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:44.544628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:44.544628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:44.544628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:44.544628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:44.545378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:44.545378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:44.545378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:44.545378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:44.545378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:44.545378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:44.545378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:44.545378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:44.545378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:44.545378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:44.545378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:28:44.545378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:28:44.545378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 92/190] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:28:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:28:44.546019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d7786-834a-4d2b-9bdc-379b6622b845 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:44.560404 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e53f29e-7b17-44b2-abcb-2ba3bf8ea9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:44.561193 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53f29e-7b17-44b2-abcb-2ba3bf8ea9d5 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:44.561482 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53f29e-7b17-44b2-abcb-2ba3bf8ea9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:44.561762 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53f29e-7b17-44b2-abcb-2ba3bf8ea9d5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:44.562927 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e53f29e-7b17-44b2-abcb-2ba3bf8ea9d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:44.564407 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e53f29e-7b17-44b2-abcb-2ba3bf8ea9d5 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:44.570685 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 99/191] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:28:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:44.570685 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e53f29e-7b17-44b2-abcb-2ba3bf8ea9d5 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:44.609443 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:44.657699 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d7786-834a-4d2b-9bdc-379b6622b845 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:44.664688 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e53f29e-7b17-44b2-abcb-2ba3bf8ea9d5 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:44.691039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e55f5b-3ff0-465c-9be7-7a64a0e4f3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:44.691708 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e55f5b-3ff0-465c-9be7-7a64a0e4f3de None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:44.691993 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e55f5b-3ff0-465c-9be7-7a64a0e4f3de None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:44.692272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e55f5b-3ff0-465c-9be7-7a64a0e4f3de None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:44.693018 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e55f5b-3ff0-465c-9be7-7a64a0e4f3de None admin] RBAC: Adding query filter params (name=reader) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:44.693325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e55f5b-3ff0-465c-9be7-7a64a0e4f3de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:44.694758 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e55f5b-3ff0-465c-9be7-7a64a0e4f3de None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:44.698449 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b92376-cc99-4acb-aa46-3efff006c3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:44.699276 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b92376-cc99-4acb-aa46-3efff006c3ae None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:44.699570 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b92376-cc99-4acb-aa46-3efff006c3ae None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:44.700272 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b92376-cc99-4acb-aa46-3efff006c3ae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:44.707201 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 93/192] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Jun 15 14:28:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:44.707201 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e55f5b-3ff0-465c-9be7-7a64a0e4f3de None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:44.716598 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b92376-cc99-4acb-aa46-3efff006c3ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:44.724769 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-d8b92376-cc99-4acb-aa46-3efff006c3ae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:28:44.724769 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:44.724769 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:44.724769 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:44.724769 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:44.724769 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:44.724769 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:44.724769 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:44.724769 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:44.724769 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:44.724769 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:44.724769 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:44.724769 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:44.724769 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:44.724769 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:44.724769 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:44.725628 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:44.725628 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:44.725628 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:44.725628 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:44.725628 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:44.725628 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:44.725628 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:44.725628 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:44.725628 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:44.725628 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:44.725628 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:44.725628 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:44.725628 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:44.725628 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:44.725628 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:44.725628 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:44.726380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:44.726380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:44.726380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:44.726380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:44.726380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:44.726380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:44.726380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:44.726380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:44.726380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:44.726380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:44.726380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:28:44.726380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:28:44.726380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 100/193] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Jun 15 14:28:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:28:44.727590 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b92376-cc99-4acb-aa46-3efff006c3ae None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:44.761559 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:44.809488 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e55f5b-3ff0-465c-9be7-7a64a0e4f3de None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:44.826707 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b92376-cc99-4acb-aa46-3efff006c3ae None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:44.828229 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:44.844312 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71f924-a8df-44ae-9235-32b8137eca6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:44.844979 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71f924-a8df-44ae-9235-32b8137eca6d None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:44.845270 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71f924-a8df-44ae-9235-32b8137eca6d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:44.845558 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71f924-a8df-44ae-9235-32b8137eca6d None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:44.846303 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf71f924-a8df-44ae-9235-32b8137eca6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:44.846616 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf71f924-a8df-44ae-9235-32b8137eca6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:44.848040 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf71f924-a8df-44ae-9235-32b8137eca6d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:44.856660 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 94/194] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:28:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:44.857615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71f924-a8df-44ae-9235-32b8137eca6d None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:44.865731 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd333f63-e254-4eb7-a7ee-d393e5951738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:44.866403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd333f63-e254-4eb7-a7ee-d393e5951738 None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:44.866681 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd333f63-e254-4eb7-a7ee-d393e5951738 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:44.866952 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd333f63-e254-4eb7-a7ee-d393e5951738 None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:44.867690 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd333f63-e254-4eb7-a7ee-d393e5951738 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:44.869123 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd333f63-e254-4eb7-a7ee-d393e5951738 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:44.889304 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 101/195] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Jun 15 14:28:44 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:28:44.891978 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd333f63-e254-4eb7-a7ee-d393e5951738 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:44.970231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71f924-a8df-44ae-9235-32b8137eca6d None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:44.988321 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:45.013871 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773a9088-564a-48da-a550-654eab187a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:45.014784 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a9088-564a-48da-a550-654eab187a4b None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:45.015105 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a9088-564a-48da-a550-654eab187a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:45.015389 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a9088-564a-48da-a550-654eab187a4b None admin] PATH_INFO: `/v3/users/4e1d412ac8d84b70a22ec80f41fded6c` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:45.017473 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773a9088-564a-48da-a550-654eab187a4b None admin] RBAC: Authorizing `identity:get_user(user_id=4e1d412ac8d84b70a22ec80f41fded6c)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:45.019113 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773a9088-564a-48da-a550-654eab187a4b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:45.021054 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 95/196] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:44 2022] GET /identity/v3/users/4e1d412ac8d84b70a22ec80f41fded6c => generated 289 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:45.026272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773a9088-564a-48da-a550-654eab187a4b None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:45.030810 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd333f63-e254-4eb7-a7ee-d393e5951738 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:45.073113 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049016ac-19f2-40d2-b103-cdfe8d13947f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:45.073113 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049016ac-19f2-40d2-b103-cdfe8d13947f None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:45.073376 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049016ac-19f2-40d2-b103-cdfe8d13947f None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:45.073619 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049016ac-19f2-40d2-b103-cdfe8d13947f None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:45.074832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049016ac-19f2-40d2-b103-cdfe8d13947f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:45.074832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049016ac-19f2-40d2-b103-cdfe8d13947f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:45.076058 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049016ac-19f2-40d2-b103-cdfe8d13947f None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:45.085875 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 102/197] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:28:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:45.086052 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049016ac-19f2-40d2-b103-cdfe8d13947f None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:45.111297 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:45.152552 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773a9088-564a-48da-a550-654eab187a4b None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:45.200744 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf868b6-5b1b-4719-a9b2-645d26f0cdc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:45.202057 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf868b6-5b1b-4719-a9b2-645d26f0cdc3 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:45.202599 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf868b6-5b1b-4719-a9b2-645d26f0cdc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:45.202879 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf868b6-5b1b-4719-a9b2-645d26f0cdc3 None admin] PATH_INFO: `/v3/users/9af0ac59a5dc489aa9147ae3a9f27101` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:45.213832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049016ac-19f2-40d2-b103-cdfe8d13947f None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:45.218607 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf868b6-5b1b-4719-a9b2-645d26f0cdc3 None admin] RBAC: Authorizing `identity:get_user(user_id=9af0ac59a5dc489aa9147ae3a9f27101)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:45.220148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf868b6-5b1b-4719-a9b2-645d26f0cdc3 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:45.221947 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 96/198] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:45 2022] GET /identity/v3/users/9af0ac59a5dc489aa9147ae3a9f27101 => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:45.222675 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf868b6-5b1b-4719-a9b2-645d26f0cdc3 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:45.256035 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f92d607-e1f7-45bd-819d-c7f5b492cfa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:45.256703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92d607-e1f7-45bd-819d-c7f5b492cfa3 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:45.257755 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92d607-e1f7-45bd-819d-c7f5b492cfa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:45.257755 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92d607-e1f7-45bd-819d-c7f5b492cfa3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:45.289102 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:45.353204 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f92d607-e1f7-45bd-819d-c7f5b492cfa3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:45.355319 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f92d607-e1f7-45bd-819d-c7f5b492cfa3 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:45.364259 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf868b6-5b1b-4719-a9b2-645d26f0cdc3 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:45.383928 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:45.408986 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4c3fe-41e8-4433-a62f-d7b96f04bd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:45.410203 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4c3fe-41e8-4433-a62f-d7b96f04bd21 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:45.410773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4c3fe-41e8-4433-a62f-d7b96f04bd21 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:45.411083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4c3fe-41e8-4433-a62f-d7b96f04bd21 None admin] PATH_INFO: `/v3/users/9af0ac59a5dc489aa9147ae3a9f27101` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:45.412520 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e4c3fe-41e8-4433-a62f-d7b96f04bd21 None admin] RBAC: Authorizing `identity:get_user(user_id=9af0ac59a5dc489aa9147ae3a9f27101)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:45.414683 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e4c3fe-41e8-4433-a62f-d7b96f04bd21 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:45.418531 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 97/199] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:45 2022] GET /identity/v3/users/9af0ac59a5dc489aa9147ae3a9f27101 => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:45.420008 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4c3fe-41e8-4433-a62f-d7b96f04bd21 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:45.517068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 103/200] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:28:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:28:45.518143 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f92d607-e1f7-45bd-819d-c7f5b492cfa3 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:45.570201 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4c3fe-41e8-4433-a62f-d7b96f04bd21 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:45.580069 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:45.608410 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c2354-8b47-44ab-9c27-16e5b38a3aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:45.609189 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c2354-8b47-44ab-9c27-16e5b38a3aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:45.609549 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c2354-8b47-44ab-9c27-16e5b38a3aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:45.609901 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c2354-8b47-44ab-9c27-16e5b38a3aa4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:45.636306 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f92d607-e1f7-45bd-819d-c7f5b492cfa3 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:45.658150 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:45.674449 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610cb57-5fe9-45f7-9866-f3109b4eb3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:45.675785 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610cb57-5fe9-45f7-9866-f3109b4eb3f2 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:45.676080 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610cb57-5fe9-45f7-9866-f3109b4eb3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:45.676361 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610cb57-5fe9-45f7-9866-f3109b4eb3f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:45.681617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0c2354-8b47-44ab-9c27-16e5b38a3aa4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:45.683267 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0c2354-8b47-44ab-9c27-16e5b38a3aa4 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:45.760160 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1610cb57-5fe9-45f7-9866-f3109b4eb3f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:45.762214 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1610cb57-5fe9-45f7-9866-f3109b4eb3f2 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:45.792905 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 98/201] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:28:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:28:45.793593 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c2354-8b47-44ab-9c27-16e5b38a3aa4 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:45.804605 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:45.878160 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 104/202] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:28:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:28:45.879078 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610cb57-5fe9-45f7-9866-f3109b4eb3f2 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:45.901985 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:45.908756 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c2354-8b47-44ab-9c27-16e5b38a3aa4 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:45.947568 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c9918-b59a-4069-be95-53b6949f4f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:45.948560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c9918-b59a-4069-be95-53b6949f4f99 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:45.948908 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c9918-b59a-4069-be95-53b6949f4f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:45.949191 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c9918-b59a-4069-be95-53b6949f4f99 None admin] PATH_INFO: `/v3/projects/8f586cc781db4f2b96f13e85c4851e5a` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:45.950395 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540c9918-b59a-4069-be95-53b6949f4f99 None admin] RBAC: Authorizing `identity:get_project(project_id=8f586cc781db4f2b96f13e85c4851e5a)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:45.952097 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540c9918-b59a-4069-be95-53b6949f4f99 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:45.954156 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 99/203] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:28:45 2022] GET /identity/v3/projects/8f586cc781db4f2b96f13e85c4851e5a => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:45.954713 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c9918-b59a-4069-be95-53b6949f4f99 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:45.987572 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610cb57-5fe9-45f7-9866-f3109b4eb3f2 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:46.023983 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b9c51-7b90-4fbc-8471-019ac863b42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:46.024843 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b9c51-7b90-4fbc-8471-019ac863b42e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:46.025128 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b9c51-7b90-4fbc-8471-019ac863b42e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:46.025410 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b9c51-7b90-4fbc-8471-019ac863b42e None admin] PATH_INFO: `/v3/projects/59af46f68a07464d897b0c71ece3f2f4` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:46.027257 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b9c51-7b90-4fbc-8471-019ac863b42e None admin] RBAC: Authorizing `identity:get_project(project_id=59af46f68a07464d897b0c71ece3f2f4)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:46.028829 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b9c51-7b90-4fbc-8471-019ac863b42e None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:46.030936 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 105/204] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:28:45 2022] GET /identity/v3/projects/59af46f68a07464d897b0c71ece3f2f4 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:46.031912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b9c51-7b90-4fbc-8471-019ac863b42e None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:46.038955 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:46.065385 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c9918-b59a-4069-be95-53b6949f4f99 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:46.102231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713db6c-460b-4b4c-8478-b8c935db4cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:46.103144 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713db6c-460b-4b4c-8478-b8c935db4cba None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:46.103435 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713db6c-460b-4b4c-8478-b8c935db4cba None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:46.103715 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713db6c-460b-4b4c-8478-b8c935db4cba None admin] PATH_INFO: `/v3/projects/59af46f68a07464d897b0c71ece3f2f4` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:46.105582 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713db6c-460b-4b4c-8478-b8c935db4cba None admin] RBAC: Authorizing `identity:get_project(project_id=59af46f68a07464d897b0c71ece3f2f4)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:46.107193 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713db6c-460b-4b4c-8478-b8c935db4cba None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:46.109234 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 100/205] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:28:45 2022] GET /identity/v3/projects/59af46f68a07464d897b0c71ece3f2f4 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:46.114720 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713db6c-460b-4b4c-8478-b8c935db4cba None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:46.129411 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:46.143087 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b9c51-7b90-4fbc-8471-019ac863b42e None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:46.183485 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b273be85-5f40-474f-8058-0e402578e826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:46.184455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273be85-5f40-474f-8058-0e402578e826 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:46.184738 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273be85-5f40-474f-8058-0e402578e826 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:46.185368 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273be85-5f40-474f-8058-0e402578e826 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:46.186885 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b273be85-5f40-474f-8058-0e402578e826 None admin] RBAC: Adding query filter params (role.id=30e8990a9ea24c4398bbe057f41400bb, scope.project.id=8f586cc781db4f2b96f13e85c4851e5a, user.id=4e1d412ac8d84b70a22ec80f41fded6c) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:46.187230 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b273be85-5f40-474f-8058-0e402578e826 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:46.188786 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b273be85-5f40-474f-8058-0e402578e826 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:46.197530 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 106/206] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:28:46 2022] GET /identity/v3/role_assignments?user.id=4e1d412ac8d84b70a22ec80f41fded6c&scope.project.id=8f586cc781db4f2b96f13e85c4851e5a&role.id=30e8990a9ea24c4398bbe057f41400bb => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:46.198125 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b273be85-5f40-474f-8058-0e402578e826 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:46.232536 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713db6c-460b-4b4c-8478-b8c935db4cba None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:46.281517 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996550e2-7ba6-4683-b89b-1d81ce28ba40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:46.282204 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996550e2-7ba6-4683-b89b-1d81ce28ba40 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:46.282491 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996550e2-7ba6-4683-b89b-1d81ce28ba40 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:46.282768 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996550e2-7ba6-4683-b89b-1d81ce28ba40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:46.283663 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996550e2-7ba6-4683-b89b-1d81ce28ba40 None admin] RBAC: Adding query filter params (role.id=ed1cfa2fcf724a4d84f5acea100930c7, scope.project.id=59af46f68a07464d897b0c71ece3f2f4, user.id=9af0ac59a5dc489aa9147ae3a9f27101) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:46.284627 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996550e2-7ba6-4683-b89b-1d81ce28ba40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:46.286141 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996550e2-7ba6-4683-b89b-1d81ce28ba40 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:46.293646 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 101/207] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:28:46 2022] GET /identity/v3/role_assignments?user.id=9af0ac59a5dc489aa9147ae3a9f27101&scope.project.id=59af46f68a07464d897b0c71ece3f2f4&role.id=ed1cfa2fcf724a4d84f5acea100930c7 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:46.355874 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b273be85-5f40-474f-8058-0e402578e826 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:46.398758 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b89e5-cfa4-4526-b189-52eabc4d46b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:46.399469 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b89e5-cfa4-4526-b189-52eabc4d46b6 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:46.399755 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b89e5-cfa4-4526-b189-52eabc4d46b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:46.400041 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b89e5-cfa4-4526-b189-52eabc4d46b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:46.400906 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b89e5-cfa4-4526-b189-52eabc4d46b6 None admin] RBAC: Adding query filter params (role.id=6b6353b49c6a41b1a157f8997fc52d9a, scope.project.id=59af46f68a07464d897b0c71ece3f2f4, user.id=9af0ac59a5dc489aa9147ae3a9f27101) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:46.401225 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b89e5-cfa4-4526-b189-52eabc4d46b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:46.403323 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b89e5-cfa4-4526-b189-52eabc4d46b6 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:46.410402 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 107/208] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:28:46 2022] GET /identity/v3/role_assignments?user.id=9af0ac59a5dc489aa9147ae3a9f27101&scope.project.id=59af46f68a07464d897b0c71ece3f2f4&role.id=6b6353b49c6a41b1a157f8997fc52d9a => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:47.318586 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232122db-91b6-484e-a6ed-addfcd202be9 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:47.319336 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232122db-91b6-484e-a6ed-addfcd202be9 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:47.319336 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232122db-91b6-484e-a6ed-addfcd202be9 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:47.320717 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 102/209] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:47.326061 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa65d7-eec1-4d4c-98ee-a52b6f121fb3 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:47.326352 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa65d7-eec1-4d4c-98ee-a52b6f121fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:47.326629 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa65d7-eec1-4d4c-98ee-a52b6f121fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:47.329796 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e15b1-1803-45a0-9297-89a51f088b8b None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:47.330086 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e15b1-1803-45a0-9297-89a51f088b8b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:47.330368 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e15b1-1803-45a0-9297-89a51f088b8b None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:47.331798 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 103/210] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:47.337739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b88eea-f113-4aa6-adda-301448dd0a04 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:47.338025 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b88eea-f113-4aa6-adda-301448dd0a04 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:47.338302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b88eea-f113-4aa6-adda-301448dd0a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:47.345670 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-14fa65d7-eec1-4d4c-98ee-a52b6f121fb3 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:47.347141 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-14fa65d7-eec1-4d4c-98ee-a52b6f121fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:47.357452 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-97b88eea-f113-4aa6-adda-301448dd0a04 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:47.359023 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-97b88eea-f113-4aa6-adda-301448dd0a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:47.360318 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:47.363275 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:47.440869 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 108/211] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:47.445347 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 104/212] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:47.465676 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa65d7-eec1-4d4c-98ee-a52b6f121fb3 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:47.467522 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b88eea-f113-4aa6-adda-301448dd0a04 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:47.590345 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b88eea-f113-4aa6-adda-301448dd0a04 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:47.604409 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa65d7-eec1-4d4c-98ee-a52b6f121fb3 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:47.628459 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30f496-68e8-4de1-8270-0a55055351e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:47.629268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30f496-68e8-4de1-8270-0a55055351e3 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:47.629550 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30f496-68e8-4de1-8270-0a55055351e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:47.629827 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30f496-68e8-4de1-8270-0a55055351e3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:47.640379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30f496-68e8-4de1-8270-0a55055351e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:47.646458 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100ae55-59c3-430c-a4a4-3879160f710a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:47.647292 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:47.647405 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100ae55-59c3-430c-a4a4-3879160f710a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:47.647567 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100ae55-59c3-430c-a4a4-3879160f710a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:47.648273 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100ae55-59c3-430c-a4a4-3879160f710a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:47.649314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-bf30f496-68e8-4de1-8270-0a55055351e3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:28:47.649314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:47.649314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:47.649314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:47.649314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:47.649314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:47.649314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:47.649314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:47.649314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:47.649314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:47.649314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:47.649314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:47.649314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:47.649314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:47.649314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:47.649314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:47.650145 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:47.650145 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:47.650145 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:47.650145 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:47.650145 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:47.650145 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:47.650145 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:47.650145 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:47.650145 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:47.650145 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:47.650145 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:47.650145 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:47.650145 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:47.650145 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:47.650145 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:47.650145 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:47.650913 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:47.650913 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:47.650913 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:47.650913 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:47.650913 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:47.650913 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:47.650913 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:47.650913 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:47.650913 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:47.650913 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:47.650913 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:28:47.650913 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:28:47.652640 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9100ae55-59c3-430c-a4a4-3879160f710a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:47.652640 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9100ae55-59c3-430c-a4a4-3879160f710a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:47.659015 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 109/213] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:28:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:47.659226 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 105/214] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:28:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:28:47.659373 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100ae55-59c3-430c-a4a4-3879160f710a None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:47.659544 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30f496-68e8-4de1-8270-0a55055351e3 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:47.673726 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:47.776346 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30f496-68e8-4de1-8270-0a55055351e3 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:47.803946 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100ae55-59c3-430c-a4a4-3879160f710a None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:47.812378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d1a361-1811-4c16-935c-2e354a5efddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:47.813131 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1a361-1811-4c16-935c-2e354a5efddb None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:47.813513 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1a361-1811-4c16-935c-2e354a5efddb None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:47.813864 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1a361-1811-4c16-935c-2e354a5efddb None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:47.814646 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d1a361-1811-4c16-935c-2e354a5efddb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:47.816244 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d1a361-1811-4c16-935c-2e354a5efddb None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:47.841110 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 106/215] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Jun 15 14:28:47 2022] POST /identity/v3/users => generated 297 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:28:47.852680 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c889a09a-34d1-40da-95b1-55c431f9b8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:47.853715 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c889a09a-34d1-40da-95b1-55c431f9b8c1 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:47.854191 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c889a09a-34d1-40da-95b1-55c431f9b8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:47.855941 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c889a09a-34d1-40da-95b1-55c431f9b8c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:47.857416 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c889a09a-34d1-40da-95b1-55c431f9b8c1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:47.857924 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c889a09a-34d1-40da-95b1-55c431f9b8c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:47.859670 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c889a09a-34d1-40da-95b1-55c431f9b8c1 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:47.870810 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 110/216] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Jun 15 14:28:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:47.877026 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d1a361-1811-4c16-935c-2e354a5efddb None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:47.948045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:48.001136 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d1a361-1811-4c16-935c-2e354a5efddb None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:48.041048 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bff352-5b81-4dc4-a148-324519acebb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:48.041930 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bff352-5b81-4dc4-a148-324519acebb4 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:48.042750 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bff352-5b81-4dc4-a148-324519acebb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:48.043088 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bff352-5b81-4dc4-a148-324519acebb4 None admin] PATH_INFO: `/v3/users/a601364422bc416093ce778ef72eacdb` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:48.058562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bff352-5b81-4dc4-a148-324519acebb4 None admin] RBAC: Authorizing `identity:get_user(user_id=a601364422bc416093ce778ef72eacdb)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:48.060080 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bff352-5b81-4dc4-a148-324519acebb4 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:48.061829 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 107/217] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:47 2022] GET /identity/v3/users/a601364422bc416093ce778ef72eacdb => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:48.067523 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c889a09a-34d1-40da-95b1-55c431f9b8c1 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:48.153132 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:48.179671 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c889a09a-34d1-40da-95b1-55c431f9b8c1 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:48.221430 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96904162-dc2d-4c9a-8b31-9a72aae2f53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:48.222321 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96904162-dc2d-4c9a-8b31-9a72aae2f53e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:48.222745 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96904162-dc2d-4c9a-8b31-9a72aae2f53e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:48.223231 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96904162-dc2d-4c9a-8b31-9a72aae2f53e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:48.300098 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96904162-dc2d-4c9a-8b31-9a72aae2f53e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:48.301866 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96904162-dc2d-4c9a-8b31-9a72aae2f53e None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:48.380755 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:48.410466 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 111/218] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:28:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:28:48.416161 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bff352-5b81-4dc4-a148-324519acebb4 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:48.543671 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bff352-5b81-4dc4-a148-324519acebb4 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:48.561345 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:48.593936 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fb7f7c-5823-440e-8951-d86ef0bdcc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:48.594899 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb7f7c-5823-440e-8951-d86ef0bdcc57 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:48.595216 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb7f7c-5823-440e-8951-d86ef0bdcc57 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:48.595498 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb7f7c-5823-440e-8951-d86ef0bdcc57 None admin] PATH_INFO: `/v3/projects/59af46f68a07464d897b0c71ece3f2f4` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:48.596831 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fb7f7c-5823-440e-8951-d86ef0bdcc57 None admin] RBAC: Authorizing `identity:get_project(project_id=59af46f68a07464d897b0c71ece3f2f4)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:48.598356 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fb7f7c-5823-440e-8951-d86ef0bdcc57 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:48.600233 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 108/219] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:28:48 2022] GET /identity/v3/projects/59af46f68a07464d897b0c71ece3f2f4 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:48.607205 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96904162-dc2d-4c9a-8b31-9a72aae2f53e None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:48.655225 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bef72-a9bc-4733-9fea-73ff924a1a96 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:48.655537 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bef72-a9bc-4733-9fea-73ff924a1a96 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:48.655790 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bef72-a9bc-4733-9fea-73ff924a1a96 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:48.657324 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 109/220] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:48.662324 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3299e-818b-4173-bfe8-12be171a240f None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:48.662558 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3299e-818b-4173-bfe8-12be171a240f None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:48.662843 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3299e-818b-4173-bfe8-12be171a240f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:48.681238 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-34c3299e-818b-4173-bfe8-12be171a240f None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:48.682441 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-34c3299e-818b-4173-bfe8-12be171a240f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:48.758449 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96904162-dc2d-4c9a-8b31-9a72aae2f53e None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:48.770245 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 110/221] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:48.780465 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e727d-8e06-4271-a8fb-dee6867669d1 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:48.780924 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e727d-8e06-4271-a8fb-dee6867669d1 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:48.781470 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e727d-8e06-4271-a8fb-dee6867669d1 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:48.783278 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 111/222] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:48.788516 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17572a22-b13a-4931-8cb7-6338aeed1efe None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:48.788967 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17572a22-b13a-4931-8cb7-6338aeed1efe None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:48.789375 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17572a22-b13a-4931-8cb7-6338aeed1efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:48.796542 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b365dccb-89b1-4656-996b-f1c4ea9f4d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:48.797336 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365dccb-89b1-4656-996b-f1c4ea9f4d80 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:48.797760 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365dccb-89b1-4656-996b-f1c4ea9f4d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:48.798125 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365dccb-89b1-4656-996b-f1c4ea9f4d80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:48.799095 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b365dccb-89b1-4656-996b-f1c4ea9f4d80 None admin] RBAC: Adding query filter params (role.id=ce0de9b9c9054826b569db895942ad26, scope.project.id=59af46f68a07464d897b0c71ece3f2f4, user.id=a601364422bc416093ce778ef72eacdb) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:48.804957 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b365dccb-89b1-4656-996b-f1c4ea9f4d80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:48.806648 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b365dccb-89b1-4656-996b-f1c4ea9f4d80 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:48.813128 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 112/223] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:28:48 2022] GET /identity/v3/role_assignments?user.id=a601364422bc416093ce778ef72eacdb&scope.project.id=59af46f68a07464d897b0c71ece3f2f4&role.id=ce0de9b9c9054826b569db895942ad26 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:48.813467 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-17572a22-b13a-4931-8cb7-6338aeed1efe None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:48.814136 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b365dccb-89b1-4656-996b-f1c4ea9f4d80 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:48.814956 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-17572a22-b13a-4931-8cb7-6338aeed1efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:48.868903 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:48.895688 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 112/224] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:48.937185 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17572a22-b13a-4931-8cb7-6338aeed1efe None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:48.943153 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b365dccb-89b1-4656-996b-f1c4ea9f4d80 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:48.986403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094edb0-fb51-4557-9c72-89b15a903c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:48.987318 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094edb0-fb51-4557-9c72-89b15a903c87 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:48.987607 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094edb0-fb51-4557-9c72-89b15a903c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:48.987883 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094edb0-fb51-4557-9c72-89b15a903c87 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:49.001473 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f094edb0-fb51-4557-9c72-89b15a903c87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:49.010407 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-f094edb0-fb51-4557-9c72-89b15a903c87 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:28:49.010407 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:49.010407 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:49.010407 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:49.010407 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:49.010407 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:49.010407 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:49.010407 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:49.010407 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:49.010407 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:49.010407 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:49.010407 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:49.010407 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:49.010407 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:49.010407 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:49.010407 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:49.011929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:49.011929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:49.011929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:49.011929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:49.011929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:49.011929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:49.011929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:49.011929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:49.011929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:49.011929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:49.011929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:49.011929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:49.011929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:49.011929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:49.011929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:49.011929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:49.012694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:49.012694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:49.012694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:49.012694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:49.012694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:49.012694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:49.012694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:49.012694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:49.012694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:49.012694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:49.012694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:28:49.012694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:28:49.012694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 113/225] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:28:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:28:49.018655 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094edb0-fb51-4557-9c72-89b15a903c87 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:49.063149 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:49.134456 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094edb0-fb51-4557-9c72-89b15a903c87 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:49.138971 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:49.173996 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1416be45-43c3-4b43-a768-5054c6372a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:49.174757 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1416be45-43c3-4b43-a768-5054c6372a65 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:49.175085 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1416be45-43c3-4b43-a768-5054c6372a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:49.175366 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1416be45-43c3-4b43-a768-5054c6372a65 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:49.176295 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1416be45-43c3-4b43-a768-5054c6372a65 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:49.176603 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1416be45-43c3-4b43-a768-5054c6372a65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:49.178385 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1416be45-43c3-4b43-a768-5054c6372a65 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:49.188190 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 114/226] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:28:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:49.193806 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1416be45-43c3-4b43-a768-5054c6372a65 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:49.249103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17572a22-b13a-4931-8cb7-6338aeed1efe None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:49.302408 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1416be45-43c3-4b43-a768-5054c6372a65 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:49.339434 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12113e95-928b-4187-9569-f89539fbfa9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:49.340735 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12113e95-928b-4187-9569-f89539fbfa9b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:49.341017 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12113e95-928b-4187-9569-f89539fbfa9b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:49.341292 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12113e95-928b-4187-9569-f89539fbfa9b None admin] PATH_INFO: `/v3/users/9af0ac59a5dc489aa9147ae3a9f27101` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:49.342624 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12113e95-928b-4187-9569-f89539fbfa9b None admin] RBAC: Authorizing `identity:get_user(user_id=9af0ac59a5dc489aa9147ae3a9f27101)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:49.344844 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12113e95-928b-4187-9569-f89539fbfa9b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:49.347026 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 115/227] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:49 2022] GET /identity/v3/users/9af0ac59a5dc489aa9147ae3a9f27101 => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:49.352920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12113e95-928b-4187-9569-f89539fbfa9b None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:49.373049 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714195c-f866-4692-ac7d-9e7c55fd8d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:49.374227 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714195c-f866-4692-ac7d-9e7c55fd8d99 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:49.374704 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714195c-f866-4692-ac7d-9e7c55fd8d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:49.375192 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714195c-f866-4692-ac7d-9e7c55fd8d99 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:49.388760 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714195c-f866-4692-ac7d-9e7c55fd8d99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:49.397680 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-d714195c-f866-4692-ac7d-9e7c55fd8d99 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:28:49.397680 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:49.397680 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:49.397680 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:49.397680 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:49.397680 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:49.397680 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:49.397680 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:49.397680 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:49.397680 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:49.397680 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:49.397680 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:49.397680 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:49.397680 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:49.397680 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:49.397680 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:49.398560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:49.398560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:49.398560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:49.398560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:49.398560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:49.398560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:49.398560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:49.398560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:49.398560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:49.398560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:49.398560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:49.398560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:49.398560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:49.398560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:49.398560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:49.398560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:49.399702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:49.399702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:49.399702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:49.399702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:49.399702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:49.399702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:49.399702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:49.399702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:49.399702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:49.399702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:49.399702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:28:49.399702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:28:49.402093 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 113/228] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Jun 15 14:28:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 465 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:28:49.404288 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:49.409148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714195c-f866-4692-ac7d-9e7c55fd8d99 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:49.489115 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12113e95-928b-4187-9569-f89539fbfa9b None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:49.525778 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7d949-6c24-42d0-8d94-d1e2d249a50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:49.527124 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7d949-6c24-42d0-8d94-d1e2d249a50a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:49.527607 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7d949-6c24-42d0-8d94-d1e2d249a50a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:49.528076 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7d949-6c24-42d0-8d94-d1e2d249a50a None admin] PATH_INFO: `/v3/projects/59af46f68a07464d897b0c71ece3f2f4` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:49.529462 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c7d949-6c24-42d0-8d94-d1e2d249a50a None admin] RBAC: Authorizing `identity:get_project(project_id=59af46f68a07464d897b0c71ece3f2f4)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:49.531129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c7d949-6c24-42d0-8d94-d1e2d249a50a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:49.533368 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 116/229] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:28:49 2022] GET /identity/v3/projects/59af46f68a07464d897b0c71ece3f2f4 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:49.540142 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7d949-6c24-42d0-8d94-d1e2d249a50a None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:49.657884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:49.662322 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714195c-f866-4692-ac7d-9e7c55fd8d99 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:49.679439 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:49.684995 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7d949-6c24-42d0-8d94-d1e2d249a50a None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:49.699244 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f178d709-8e15-4aa8-8fc9-fda79fd061ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:49.699923 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178d709-8e15-4aa8-8fc9-fda79fd061ae None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:49.700209 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178d709-8e15-4aa8-8fc9-fda79fd061ae None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:49.700489 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178d709-8e15-4aa8-8fc9-fda79fd061ae None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:49.701250 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f178d709-8e15-4aa8-8fc9-fda79fd061ae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:49.701561 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f178d709-8e15-4aa8-8fc9-fda79fd061ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:49.704658 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f178d709-8e15-4aa8-8fc9-fda79fd061ae None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:49.715017 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 114/230] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:28:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:49.717381 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f178d709-8e15-4aa8-8fc9-fda79fd061ae None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:49.808239 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19026780-1f08-4463-b48f-ee14b4b459c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:49.809139 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19026780-1f08-4463-b48f-ee14b4b459c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:49.809423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19026780-1f08-4463-b48f-ee14b4b459c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:49.809697 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19026780-1f08-4463-b48f-ee14b4b459c0 None admin] PATH_INFO: `/v3/projects/59af46f68a07464d897b0c71ece3f2f4/users/9af0ac59a5dc489aa9147ae3a9f27101/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:49.811682 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19026780-1f08-4463-b48f-ee14b4b459c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=59af46f68a07464d897b0c71ece3f2f4, user_id=9af0ac59a5dc489aa9147ae3a9f27101, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:49.813222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19026780-1f08-4463-b48f-ee14b4b459c0 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:49.820267 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f178d709-8e15-4aa8-8fc9-fda79fd061ae None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:49.846543 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 117/231] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Jun 15 14:28:49 2022] PUT /identity/v3/projects/59af46f68a07464d897b0c71ece3f2f4/users/9af0ac59a5dc489aa9147ae3a9f27101/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:28:49.858222 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a9494-b7e4-4b5c-b192-68d0b8bd956e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:49.859267 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a9494-b7e4-4b5c-b192-68d0b8bd956e None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:49.859849 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a9494-b7e4-4b5c-b192-68d0b8bd956e None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:49.860150 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a9494-b7e4-4b5c-b192-68d0b8bd956e None admin] PATH_INFO: `/v3/users/9af0ac59a5dc489aa9147ae3a9f27101` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:49.862185 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8a9494-b7e4-4b5c-b192-68d0b8bd956e None admin] RBAC: Authorizing `identity:get_user(user_id=9af0ac59a5dc489aa9147ae3a9f27101)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:49.864727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8a9494-b7e4-4b5c-b192-68d0b8bd956e None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:49.866960 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 115/232] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:49 2022] GET /identity/v3/users/9af0ac59a5dc489aa9147ae3a9f27101 => generated 283 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:49.873203 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19026780-1f08-4463-b48f-ee14b4b459c0 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:50.034342 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19026780-1f08-4463-b48f-ee14b4b459c0 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:50.082062 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e95c62-434f-40da-83a0-fb93c46e12ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:50.082907 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e95c62-434f-40da-83a0-fb93c46e12ee None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:50.083219 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e95c62-434f-40da-83a0-fb93c46e12ee None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:50.083499 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e95c62-434f-40da-83a0-fb93c46e12ee None admin] PATH_INFO: `/v3/projects/59af46f68a07464d897b0c71ece3f2f4` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:50.084758 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e95c62-434f-40da-83a0-fb93c46e12ee None admin] RBAC: Authorizing `identity:get_project(project_id=59af46f68a07464d897b0c71ece3f2f4)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:50.086230 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e95c62-434f-40da-83a0-fb93c46e12ee None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:50.088265 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 118/233] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:28:49 2022] GET /identity/v3/projects/59af46f68a07464d897b0c71ece3f2f4 => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:50.094699 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a9494-b7e4-4b5c-b192-68d0b8bd956e None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:50.155277 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:50.238134 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a9494-b7e4-4b5c-b192-68d0b8bd956e None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:50.273843 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06894f9-f386-4d90-87ee-916b6404f249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:50.274743 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06894f9-f386-4d90-87ee-916b6404f249 None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:50.275057 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06894f9-f386-4d90-87ee-916b6404f249 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:50.275377 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06894f9-f386-4d90-87ee-916b6404f249 None admin] PATH_INFO: `/v3/projects/59af46f68a07464d897b0c71ece3f2f4/users/9af0ac59a5dc489aa9147ae3a9f27101/roles/6b6353b49c6a41b1a157f8997fc52d9a` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:50.278219 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06894f9-f386-4d90-87ee-916b6404f249 None admin] RBAC: Authorizing `identity:create_grant(project_id=59af46f68a07464d897b0c71ece3f2f4, user_id=9af0ac59a5dc489aa9147ae3a9f27101, role_id=6b6353b49c6a41b1a157f8997fc52d9a)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:50.279712 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06894f9-f386-4d90-87ee-916b6404f249 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:50.299075 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 116/234] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Jun 15 14:28:50 2022] PUT /identity/v3/projects/59af46f68a07464d897b0c71ece3f2f4/users/9af0ac59a5dc489aa9147ae3a9f27101/roles/6b6353b49c6a41b1a157f8997fc52d9a => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:28:50.736793 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910efa31-deed-4d7e-9b9e-2456836584f1 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:50.737280 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910efa31-deed-4d7e-9b9e-2456836584f1 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:50.737352 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910efa31-deed-4d7e-9b9e-2456836584f1 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:50.739027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 119/235] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:50.744311 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c3eda-fe38-4a7f-857a-ae34d756b03c None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:50.744768 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c3eda-fe38-4a7f-857a-ae34d756b03c None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:50.744842 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c3eda-fe38-4a7f-857a-ae34d756b03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:50.763323 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-340c3eda-fe38-4a7f-857a-ae34d756b03c None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:50.764530 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-340c3eda-fe38-4a7f-857a-ae34d756b03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:50.817499 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9bbd82-d30c-48d9-94a9-22305fbbc1a0 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:50.817823 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9bbd82-d30c-48d9-94a9-22305fbbc1a0 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:50.818068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9bbd82-d30c-48d9-94a9-22305fbbc1a0 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:50.819761 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 120/236] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:50.826362 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6532b-c61e-461c-81d4-fdeff0a6835c None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:50.826654 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6532b-c61e-461c-81d4-fdeff0a6835c None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:50.826934 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6532b-c61e-461c-81d4-fdeff0a6835c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:50.838352 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 117/237] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:50.859996 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-faa6532b-c61e-461c-81d4-fdeff0a6835c None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:50.861197 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-faa6532b-c61e-461c-81d4-fdeff0a6835c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:50.861623 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c3eda-fe38-4a7f-857a-ae34d756b03c None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:50.898772 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:50.899097 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:50.984920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 121/238] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:28:51.009860 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6532b-c61e-461c-81d4-fdeff0a6835c None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:51.033264 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c3eda-fe38-4a7f-857a-ae34d756b03c None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:51.083882 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1da5bc-625e-441a-88f0-7bbb8c8466fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:51.084692 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1da5bc-625e-441a-88f0-7bbb8c8466fe None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:51.084970 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1da5bc-625e-441a-88f0-7bbb8c8466fe None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:51.085239 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1da5bc-625e-441a-88f0-7bbb8c8466fe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:51.109432 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1da5bc-625e-441a-88f0-7bbb8c8466fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:51.117115 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-0e1da5bc-625e-441a-88f0-7bbb8c8466fe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:28:51.117115 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:51.117115 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:51.117115 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:51.117115 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:51.117115 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:51.117115 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:51.117115 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:51.117115 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:51.117115 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:51.117115 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:51.117115 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:51.117115 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:51.117115 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:51.117115 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:51.117115 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:51.117992 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:51.117992 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:51.117992 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:51.117992 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:51.117992 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:51.117992 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:51.117992 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:51.117992 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:51.117992 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:51.117992 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:51.117992 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:51.117992 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:51.117992 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:51.117992 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:51.117992 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:51.117992 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:51.118751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:51.118751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:51.118751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:51.118751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:51.118751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:51.118751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:51.118751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:51.118751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:51.118751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:51.118751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:51.118751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:28:51.118751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:28:51.118751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 118/239] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:28:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:28:51.125992 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1da5bc-625e-441a-88f0-7bbb8c8466fe None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:51.189610 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6532b-c61e-461c-81d4-fdeff0a6835c None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:51.245969 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:51.264816 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b31bc-2f30-4e84-87fe-6981002db187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:51.265626 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b31bc-2f30-4e84-87fe-6981002db187 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:51.265906 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b31bc-2f30-4e84-87fe-6981002db187 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:51.266177 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b31bc-2f30-4e84-87fe-6981002db187 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:51.267640 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620b31bc-2f30-4e84-87fe-6981002db187 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:51.268433 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1da5bc-625e-441a-88f0-7bbb8c8466fe None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:51.269099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620b31bc-2f30-4e84-87fe-6981002db187 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:51.283296 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 122/240] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:28:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:51.293143 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b31bc-2f30-4e84-87fe-6981002db187 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:51.327213 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd8da4-3e6a-4c36-8395-0c98fd9388fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:51.328394 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd8da4-3e6a-4c36-8395-0c98fd9388fa None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:51.328880 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd8da4-3e6a-4c36-8395-0c98fd9388fa None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:51.329351 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd8da4-3e6a-4c36-8395-0c98fd9388fa None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:51.330293 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd8da4-3e6a-4c36-8395-0c98fd9388fa None admin] RBAC: Adding query filter params (name=reader) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:51.330801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd8da4-3e6a-4c36-8395-0c98fd9388fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:51.332460 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd8da4-3e6a-4c36-8395-0c98fd9388fa None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:51.336922 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:51.343709 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 119/241] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Jun 15 14:28:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:51.349473 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5704a67-40e3-4408-bec7-02c98750fa38 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:51.350141 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5704a67-40e3-4408-bec7-02c98750fa38 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:51.350639 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5704a67-40e3-4408-bec7-02c98750fa38 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:51.352220 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 120/242] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:51.353130 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5704a67-40e3-4408-bec7-02c98750fa38 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:51.438656 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b31bc-2f30-4e84-87fe-6981002db187 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:51.479084 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f042a-d0fd-442e-aa86-582db3b1db37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:51.480338 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f042a-d0fd-442e-aa86-582db3b1db37 None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:51.480619 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f042a-d0fd-442e-aa86-582db3b1db37 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:51.480891 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f042a-d0fd-442e-aa86-582db3b1db37 None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:51.481734 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f042a-d0fd-442e-aa86-582db3b1db37 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:51.483305 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f042a-d0fd-442e-aa86-582db3b1db37 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:51.490294 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5704a67-40e3-4408-bec7-02c98750fa38 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:51.510176 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 123/243] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Jun 15 14:28:51 2022] POST /identity/v3/users => generated 293 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:28:51.519204 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140f533-e71c-4553-8b63-00dbc899277b None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:51.519204 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140f533-e71c-4553-8b63-00dbc899277b None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:51.519204 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140f533-e71c-4553-8b63-00dbc899277b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:51.556190 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-4140f533-e71c-4553-8b63-00dbc899277b None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:51.557423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-4140f533-e71c-4553-8b63-00dbc899277b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:51.566097 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2277d-c64d-42ec-8735-c00010f7125a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:51.567596 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2277d-c64d-42ec-8735-c00010f7125a None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:51.568078 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2277d-c64d-42ec-8735-c00010f7125a None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:51.568592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2277d-c64d-42ec-8735-c00010f7125a None admin] PATH_INFO: `/v3/users/56fb5916a497403c8cead2cbebe4d131` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:51.596642 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:51.599996 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea2277d-c64d-42ec-8735-c00010f7125a None admin] RBAC: Authorizing `identity:get_user(user_id=56fb5916a497403c8cead2cbebe4d131)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:51.601789 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea2277d-c64d-42ec-8735-c00010f7125a None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:51.604154 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 121/244] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:51 2022] GET /identity/v3/users/56fb5916a497403c8cead2cbebe4d131 => generated 297 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:51.613958 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2277d-c64d-42ec-8735-c00010f7125a None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:51.725030 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2277d-c64d-42ec-8735-c00010f7125a None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:51.758700 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938cc884-5dd7-43da-811a-9491df5e4f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:51.759542 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938cc884-5dd7-43da-811a-9491df5e4f0e None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:51.759906 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 124/245] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:51.760328 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938cc884-5dd7-43da-811a-9491df5e4f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:51.761804 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938cc884-5dd7-43da-811a-9491df5e4f0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:51.811193 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4140f533-e71c-4553-8b63-00dbc899277b None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:51.834633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:51.839915 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:51.841559 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938cc884-5dd7-43da-811a-9491df5e4f0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:51.847265 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938cc884-5dd7-43da-811a-9491df5e4f0e None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:51.991759 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4140f533-e71c-4553-8b63-00dbc899277b None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:51.997074 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 122/246] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:28:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:28:52.003275 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938cc884-5dd7-43da-811a-9491df5e4f0e None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:52.054625 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264b0e2-b702-4bd7-b08c-b8dbf05380ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:52.059647 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264b0e2-b702-4bd7-b08c-b8dbf05380ef None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:52.060145 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264b0e2-b702-4bd7-b08c-b8dbf05380ef None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:52.060607 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264b0e2-b702-4bd7-b08c-b8dbf05380ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:52.084161 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e264b0e2-b702-4bd7-b08c-b8dbf05380ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:52.100800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-e264b0e2-b702-4bd7-b08c-b8dbf05380ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:28:52.100800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:52.100800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:52.100800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:52.100800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:52.100800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:52.100800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:52.100800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:52.100800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:52.100800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:52.100800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:52.100800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:52.100800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:52.100800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:52.100800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:52.100800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:52.101784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:52.101784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:52.101784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:52.101784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:52.101784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:52.101784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:52.101784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:52.101784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:52.101784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:52.101784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:52.101784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:52.101784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:52.101784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:52.101784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:52.101784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:52.101784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:52.102532 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:52.102532 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:52.102532 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:52.102532 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:52.102532 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:52.102532 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:52.102532 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:52.102532 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:52.102532 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:52.102532 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:52.102532 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:28:52.102532 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:28:52.106010 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 125/247] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:28:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:28:52.112345 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264b0e2-b702-4bd7-b08c-b8dbf05380ef None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:52.124174 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:52.127174 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938cc884-5dd7-43da-811a-9491df5e4f0e None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:52.172480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3837d-c61e-46ee-8a90-8749103ab517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:52.172480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3837d-c61e-46ee-8a90-8749103ab517 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:52.172480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3837d-c61e-46ee-8a90-8749103ab517 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:52.172820 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3837d-c61e-46ee-8a90-8749103ab517 None admin] PATH_INFO: `/v3/projects/59af46f68a07464d897b0c71ece3f2f4` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:52.173837 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe3837d-c61e-46ee-8a90-8749103ab517 None admin] RBAC: Authorizing `identity:get_project(project_id=59af46f68a07464d897b0c71ece3f2f4)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:52.176542 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe3837d-c61e-46ee-8a90-8749103ab517 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:52.177398 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 123/248] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:28:52 2022] GET /identity/v3/projects/59af46f68a07464d897b0c71ece3f2f4 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:52.185988 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3837d-c61e-46ee-8a90-8749103ab517 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:52.237693 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:52.284022 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264b0e2-b702-4bd7-b08c-b8dbf05380ef None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:52.316137 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3837d-c61e-46ee-8a90-8749103ab517 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:52.325452 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a7f0b2-8380-4757-a38d-0fa40b13b9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:52.326431 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7f0b2-8380-4757-a38d-0fa40b13b9c9 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:52.326937 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7f0b2-8380-4757-a38d-0fa40b13b9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:52.327527 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7f0b2-8380-4757-a38d-0fa40b13b9c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:52.328493 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a7f0b2-8380-4757-a38d-0fa40b13b9c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:52.329106 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a7f0b2-8380-4757-a38d-0fa40b13b9c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:52.330681 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a7f0b2-8380-4757-a38d-0fa40b13b9c9 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:52.343304 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 126/249] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Jun 15 14:28:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:52.350262 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a7f0b2-8380-4757-a38d-0fa40b13b9c9 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:52.360678 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b566d29-2527-4df2-a866-b1e953e37ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:52.361343 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b566d29-2527-4df2-a866-b1e953e37ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:52.361630 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b566d29-2527-4df2-a866-b1e953e37ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:52.361904 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b566d29-2527-4df2-a866-b1e953e37ad1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:52.362742 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b566d29-2527-4df2-a866-b1e953e37ad1 None admin] RBAC: Adding query filter params (role.id=30e8990a9ea24c4398bbe057f41400bb, scope.project.id=59af46f68a07464d897b0c71ece3f2f4, user.id=56fb5916a497403c8cead2cbebe4d131) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:52.363090 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b566d29-2527-4df2-a866-b1e953e37ad1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:52.364711 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b566d29-2527-4df2-a866-b1e953e37ad1 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:52.373551 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 124/250] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:28:52 2022] GET /identity/v3/role_assignments?user.id=56fb5916a497403c8cead2cbebe4d131&scope.project.id=59af46f68a07464d897b0c71ece3f2f4&role.id=30e8990a9ea24c4398bbe057f41400bb => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:52.488739 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a7f0b2-8380-4757-a38d-0fa40b13b9c9 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:52.522320 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:52.528204 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d564b732-17c4-4bf2-8c35-dae0889550ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:52.528990 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564b732-17c4-4bf2-8c35-dae0889550ce None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:52.529271 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564b732-17c4-4bf2-8c35-dae0889550ce None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:52.529550 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564b732-17c4-4bf2-8c35-dae0889550ce None admin] PATH_INFO: `/v3/users/a601364422bc416093ce778ef72eacdb` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:52.530834 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d564b732-17c4-4bf2-8c35-dae0889550ce None admin] RBAC: Authorizing `identity:get_user(user_id=a601364422bc416093ce778ef72eacdb)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:52.532578 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d564b732-17c4-4bf2-8c35-dae0889550ce None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:52.534790 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 127/251] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:52 2022] GET /identity/v3/users/a601364422bc416093ce778ef72eacdb => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:52.540763 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b566d29-2527-4df2-a866-b1e953e37ad1 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:52.586547 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:52.672326 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b566d29-2527-4df2-a866-b1e953e37ad1 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:52.708650 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf22c95-6924-4440-91d0-ca55104dc2bb None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:52.709414 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf22c95-6924-4440-91d0-ca55104dc2bb None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:52.710330 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf22c95-6924-4440-91d0-ca55104dc2bb None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:52.712482 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 128/252] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:52.716613 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09472501-d069-4e99-8b74-b0de5ba459b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:52.717546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09472501-d069-4e99-8b74-b0de5ba459b4 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:52.717837 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09472501-d069-4e99-8b74-b0de5ba459b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:52.718115 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09472501-d069-4e99-8b74-b0de5ba459b4 None admin] PATH_INFO: `/v3/projects/59af46f68a07464d897b0c71ece3f2f4` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:52.719336 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b01f4-3710-4319-997c-7a569c05286a None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:52.719441 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09472501-d069-4e99-8b74-b0de5ba459b4 None admin] RBAC: Authorizing `identity:get_project(project_id=59af46f68a07464d897b0c71ece3f2f4)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:52.719905 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b01f4-3710-4319-997c-7a569c05286a None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:52.720371 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b01f4-3710-4319-997c-7a569c05286a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:52.720826 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09472501-d069-4e99-8b74-b0de5ba459b4 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:52.725066 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 125/253] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:28:52 2022] GET /identity/v3/projects/59af46f68a07464d897b0c71ece3f2f4 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:52.731048 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09472501-d069-4e99-8b74-b0de5ba459b4 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:52.757290 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-d62b01f4-3710-4319-997c-7a569c05286a None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:52.760560 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d62b01f4-3710-4319-997c-7a569c05286a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:52.890349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 129/254] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:28:52.908433 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:52.913188 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09472501-d069-4e99-8b74-b0de5ba459b4 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:52.918624 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b01f4-3710-4319-997c-7a569c05286a None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:53.000286 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf2635-4263-4bad-844d-9475dbcddd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:53.001203 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf2635-4263-4bad-844d-9475dbcddd8f None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:53.001487 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf2635-4263-4bad-844d-9475dbcddd8f None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:53.001760 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf2635-4263-4bad-844d-9475dbcddd8f None admin] PATH_INFO: `/v3/projects/59af46f68a07464d897b0c71ece3f2f4/users/a601364422bc416093ce778ef72eacdb/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:53.003800 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cf2635-4263-4bad-844d-9475dbcddd8f None admin] RBAC: Authorizing `identity:create_grant(project_id=59af46f68a07464d897b0c71ece3f2f4, user_id=a601364422bc416093ce778ef72eacdb, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:53.005289 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cf2635-4263-4bad-844d-9475dbcddd8f None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:53.030416 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b01f4-3710-4319-997c-7a569c05286a None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:53.030859 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 126/255] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Jun 15 14:28:52 2022] PUT /identity/v3/projects/59af46f68a07464d897b0c71ece3f2f4/users/a601364422bc416093ce778ef72eacdb/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:28:53.062974 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:53.075831 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a87365b-c7c5-4a4e-a196-56c28b05147b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:53.076663 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87365b-c7c5-4a4e-a196-56c28b05147b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:53.076950 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87365b-c7c5-4a4e-a196-56c28b05147b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:53.077228 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87365b-c7c5-4a4e-a196-56c28b05147b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:53.095617 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a87365b-c7c5-4a4e-a196-56c28b05147b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:53.103521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-6a87365b-c7c5-4a4e-a196-56c28b05147b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:28:53.103521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:53.103521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:53.103521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:53.103521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:53.103521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:53.103521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:53.103521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:53.103521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:53.103521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:53.103521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:53.103521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:53.103521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:53.103521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:53.103521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:53.103521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:53.104450 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:53.104450 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:53.104450 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:53.104450 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:53.104450 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:53.104450 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:53.104450 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:53.104450 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:53.104450 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:53.104450 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:53.104450 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:53.104450 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:53.104450 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:53.104450 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:53.104450 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:53.104450 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:53.105269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:53.105269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:53.105269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:53.105269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:53.105269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:53.105269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:53.105269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:53.105269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:53.105269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:53.105269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:53.105269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:28:53.105269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:28:53.109377 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 130/256] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:28:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:28:53.110768 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf2635-4263-4bad-844d-9475dbcddd8f None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:53.276730 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337a46a-2c86-4751-b99f-52622187f3c4 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:53.277063 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337a46a-2c86-4751-b99f-52622187f3c4 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:53.277240 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337a46a-2c86-4751-b99f-52622187f3c4 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:53.278931 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 131/257] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:53.288464 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3068ad-1697-4f21-a0f5-c15ec8dffc15 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:53.288757 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3068ad-1697-4f21-a0f5-c15ec8dffc15 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:53.289037 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3068ad-1697-4f21-a0f5-c15ec8dffc15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:53.290897 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf2635-4263-4bad-844d-9475dbcddd8f None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:53.330561 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd39921-71a8-4c03-9285-c17a54085c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:53.331540 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-ce3068ad-1697-4f21-a0f5-c15ec8dffc15 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:53.331870 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd39921-71a8-4c03-9285-c17a54085c6d None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:53.332350 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd39921-71a8-4c03-9285-c17a54085c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:53.332761 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ce3068ad-1697-4f21-a0f5-c15ec8dffc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:53.333064 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd39921-71a8-4c03-9285-c17a54085c6d None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:53.334030 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd39921-71a8-4c03-9285-c17a54085c6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:53.334626 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd39921-71a8-4c03-9285-c17a54085c6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:53.336728 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd39921-71a8-4c03-9285-c17a54085c6d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:53.347087 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 127/258] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:28:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:53.360451 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd39921-71a8-4c03-9285-c17a54085c6d None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:53.403818 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:53.456910 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 132/259] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:53.503014 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd39921-71a8-4c03-9285-c17a54085c6d None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:53.508079 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3068ad-1697-4f21-a0f5-c15ec8dffc15 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:53.559844 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11746ea2-98d0-41d5-a472-475914b17bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:53.560730 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11746ea2-98d0-41d5-a472-475914b17bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:53.561023 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11746ea2-98d0-41d5-a472-475914b17bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:53.561973 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11746ea2-98d0-41d5-a472-475914b17bb3 None admin] PATH_INFO: `/v3/users/9af0ac59a5dc489aa9147ae3a9f27101` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:53.571019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11746ea2-98d0-41d5-a472-475914b17bb3 None admin] RBAC: Authorizing `identity:get_user(user_id=9af0ac59a5dc489aa9147ae3a9f27101)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:53.571019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11746ea2-98d0-41d5-a472-475914b17bb3 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:53.571019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 128/260] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:53 2022] GET /identity/v3/users/9af0ac59a5dc489aa9147ae3a9f27101 => generated 283 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:53.576581 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11746ea2-98d0-41d5-a472-475914b17bb3 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:53.633275 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3068ad-1697-4f21-a0f5-c15ec8dffc15 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:53.669286 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae14fed0-1acb-45e2-9995-afe0d6f76682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:53.670452 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae14fed0-1acb-45e2-9995-afe0d6f76682 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:53.670952 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae14fed0-1acb-45e2-9995-afe0d6f76682 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:53.671624 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae14fed0-1acb-45e2-9995-afe0d6f76682 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:53.683196 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:53.687395 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae14fed0-1acb-45e2-9995-afe0d6f76682 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:53.700855 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-ae14fed0-1acb-45e2-9995-afe0d6f76682 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:28:53.700855 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:53.700855 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:53.700855 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:53.700855 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:53.700855 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:53.700855 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:53.700855 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:53.700855 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:53.700855 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:53.700855 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:53.700855 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:53.700855 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:53.700855 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:53.700855 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:53.700855 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:53.701752 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:53.701752 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:53.701752 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:53.701752 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:53.701752 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:53.701752 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:53.701752 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:53.701752 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:53.701752 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:53.701752 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:53.701752 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:53.701752 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:53.701752 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:53.701752 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:53.701752 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:53.701752 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:53.702521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:53.702521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:53.702521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:53.702521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:53.702521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:53.702521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:53.702521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:53.702521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:53.702521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:53.702521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:53.702521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:28:53.702521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:28:53.709830 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 133/261] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Jun 15 14:28:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:28:53.715802 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae14fed0-1acb-45e2-9995-afe0d6f76682 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:53.727357 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:53.738544 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11746ea2-98d0-41d5-a472-475914b17bb3 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:53.798634 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae263a-5dd7-40d0-a693-8c543ac69b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:53.799384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae263a-5dd7-40d0-a693-8c543ac69b65 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:53.799678 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae263a-5dd7-40d0-a693-8c543ac69b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:53.799959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae263a-5dd7-40d0-a693-8c543ac69b65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:53.877017 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbae263a-5dd7-40d0-a693-8c543ac69b65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:53.878486 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbae263a-5dd7-40d0-a693-8c543ac69b65 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:53.961023 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae14fed0-1acb-45e2-9995-afe0d6f76682 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:53.978714 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:53.988871 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 129/262] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:28:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 14:28:53.995674 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae263a-5dd7-40d0-a693-8c543ac69b65 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:54.009407 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85fc788-4f2a-479a-bf59-023bece8515f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:54.010251 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fc788-4f2a-479a-bf59-023bece8515f None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:54.010644 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fc788-4f2a-479a-bf59-023bece8515f None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:54.015319 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fc788-4f2a-479a-bf59-023bece8515f None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:54.016299 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85fc788-4f2a-479a-bf59-023bece8515f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:54.016721 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85fc788-4f2a-479a-bf59-023bece8515f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:54.018503 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85fc788-4f2a-479a-bf59-023bece8515f None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:54.036368 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 134/263] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:28:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:54.047302 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85fc788-4f2a-479a-bf59-023bece8515f None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:54.110128 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae263a-5dd7-40d0-a693-8c543ac69b65 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:54.131315 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:54.166220 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0876cf-c587-4a4f-94a8-5da110623b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:54.169092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0876cf-c587-4a4f-94a8-5da110623b1a None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:54.170836 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0876cf-c587-4a4f-94a8-5da110623b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:54.171340 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0876cf-c587-4a4f-94a8-5da110623b1a None admin] PATH_INFO: `/v3/projects/59af46f68a07464d897b0c71ece3f2f4` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:54.175881 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0876cf-c587-4a4f-94a8-5da110623b1a None admin] RBAC: Authorizing `identity:get_project(project_id=59af46f68a07464d897b0c71ece3f2f4)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:54.178179 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0876cf-c587-4a4f-94a8-5da110623b1a None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:54.190504 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 130/264] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:28:53 2022] GET /identity/v3/projects/59af46f68a07464d897b0c71ece3f2f4 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:54.209082 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0876cf-c587-4a4f-94a8-5da110623b1a None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:54.272039 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85fc788-4f2a-479a-bf59-023bece8515f None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:54.278674 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:54.327381 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0876cf-c587-4a4f-94a8-5da110623b1a None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:54.364722 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf01a1e-ff93-44ac-b2f4-f9427449980a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:54.365776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf01a1e-ff93-44ac-b2f4-f9427449980a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:54.366218 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf01a1e-ff93-44ac-b2f4-f9427449980a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:54.366503 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf01a1e-ff93-44ac-b2f4-f9427449980a None admin] PATH_INFO: `/v3/users/9af0ac59a5dc489aa9147ae3a9f27101` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:54.371022 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf01a1e-ff93-44ac-b2f4-f9427449980a None admin] RBAC: Authorizing `identity:get_user(user_id=9af0ac59a5dc489aa9147ae3a9f27101)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:54.371022 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf01a1e-ff93-44ac-b2f4-f9427449980a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:54.371473 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 135/265] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:54 2022] GET /identity/v3/users/9af0ac59a5dc489aa9147ae3a9f27101 => generated 283 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:54.372130 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e2d24-c8d5-41ed-8ac9-4d093d8cfca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:54.373014 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e2d24-c8d5-41ed-8ac9-4d093d8cfca8 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:54.373481 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e2d24-c8d5-41ed-8ac9-4d093d8cfca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:54.373939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e2d24-c8d5-41ed-8ac9-4d093d8cfca8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:54.375162 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677e2d24-c8d5-41ed-8ac9-4d093d8cfca8 None admin] RBAC: Adding query filter params (role.id=ed1cfa2fcf724a4d84f5acea100930c7, scope.project.id=59af46f68a07464d897b0c71ece3f2f4, user.id=9af0ac59a5dc489aa9147ae3a9f27101) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:54.375669 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677e2d24-c8d5-41ed-8ac9-4d093d8cfca8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:54.376779 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf01a1e-ff93-44ac-b2f4-f9427449980a None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:54.377569 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677e2d24-c8d5-41ed-8ac9-4d093d8cfca8 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:54.400467 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 131/266] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:28:54 2022] GET /identity/v3/role_assignments?user.id=9af0ac59a5dc489aa9147ae3a9f27101&scope.project.id=59af46f68a07464d897b0c71ece3f2f4&role.id=ed1cfa2fcf724a4d84f5acea100930c7 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:54.527889 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf01a1e-ff93-44ac-b2f4-f9427449980a None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:54.545697 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:54.578243 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f12fad-bc4d-4c55-8421-e88ec9120faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:54.578917 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f12fad-bc4d-4c55-8421-e88ec9120faf None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:54.579230 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f12fad-bc4d-4c55-8421-e88ec9120faf None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:54.579514 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f12fad-bc4d-4c55-8421-e88ec9120faf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:54.679816 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f12fad-bc4d-4c55-8421-e88ec9120faf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:54.681231 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f12fad-bc4d-4c55-8421-e88ec9120faf None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:54.818010 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 136/267] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:28:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:28:54.823858 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e2d24-c8d5-41ed-8ac9-4d093d8cfca8 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:54.940278 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e2d24-c8d5-41ed-8ac9-4d093d8cfca8 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:54.969002 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:54.983398 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a4417-e95c-452f-a76e-22869481ebd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:54.984276 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a4417-e95c-452f-a76e-22869481ebd8 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:54.984565 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a4417-e95c-452f-a76e-22869481ebd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:54.984844 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a4417-e95c-452f-a76e-22869481ebd8 None admin] PATH_INFO: `/v3/projects/59af46f68a07464d897b0c71ece3f2f4` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:54.991880 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a4417-e95c-452f-a76e-22869481ebd8 None admin] RBAC: Authorizing `identity:get_project(project_id=59af46f68a07464d897b0c71ece3f2f4)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:54.993394 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a4417-e95c-452f-a76e-22869481ebd8 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:54.995880 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 132/268] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:28:54 2022] GET /identity/v3/projects/59af46f68a07464d897b0c71ece3f2f4 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:55.002277 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f12fad-bc4d-4c55-8421-e88ec9120faf None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:55.016011 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:55.080983 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14534021-a787-4a77-8d19-83f86023e1ac None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:55.081317 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14534021-a787-4a77-8d19-83f86023e1ac None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:55.081544 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14534021-a787-4a77-8d19-83f86023e1ac None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:55.082946 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 133/269] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:28:55.088126 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43419513-ed92-4315-90cf-73ca4aa30451 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:55.088418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43419513-ed92-4315-90cf-73ca4aa30451 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:55.088698 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43419513-ed92-4315-90cf-73ca4aa30451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:55.109633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-43419513-ed92-4315-90cf-73ca4aa30451 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:55.110872 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-43419513-ed92-4315-90cf-73ca4aa30451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:55.156595 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f12fad-bc4d-4c55-8421-e88ec9120faf None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:55.201856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e842b-8ecf-4261-98b8-2a73b54b8392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:55.202872 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e842b-8ecf-4261-98b8-2a73b54b8392 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:55.203463 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e842b-8ecf-4261-98b8-2a73b54b8392 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:55.204101 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e842b-8ecf-4261-98b8-2a73b54b8392 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:55.205172 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238e842b-8ecf-4261-98b8-2a73b54b8392 None admin] RBAC: Adding query filter params (role.id=6b6353b49c6a41b1a157f8997fc52d9a, scope.project.id=59af46f68a07464d897b0c71ece3f2f4, user.id=9af0ac59a5dc489aa9147ae3a9f27101) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:55.205674 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238e842b-8ecf-4261-98b8-2a73b54b8392 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:55.207530 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238e842b-8ecf-4261-98b8-2a73b54b8392 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:55.214155 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 134/270] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:55.214745 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 137/271] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:28:55 2022] GET /identity/v3/role_assignments?user.id=9af0ac59a5dc489aa9147ae3a9f27101&scope.project.id=59af46f68a07464d897b0c71ece3f2f4&role.id=6b6353b49c6a41b1a157f8997fc52d9a => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:55.258485 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43419513-ed92-4315-90cf-73ca4aa30451 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:55.380707 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43419513-ed92-4315-90cf-73ca4aa30451 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:55.404715 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:55.420458 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6642bb92-7704-4aa4-99a4-fda39789f262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:55.421272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642bb92-7704-4aa4-99a4-fda39789f262 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:55.421990 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642bb92-7704-4aa4-99a4-fda39789f262 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:55.422271 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642bb92-7704-4aa4-99a4-fda39789f262 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:55.435762 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6642bb92-7704-4aa4-99a4-fda39789f262 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:55.437915 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a83447-8a62-44e8-840e-ead907f96257 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:55.438204 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a83447-8a62-44e8-840e-ead907f96257 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:55.438555 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a83447-8a62-44e8-840e-ead907f96257 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:55.440008 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 138/272] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:55.444476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-6642bb92-7704-4aa4-99a4-fda39789f262 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:28:55.444476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:55.444476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:55.444476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:55.444476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:55.444476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:55.444476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:55.444476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:55.444476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:55.444476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:55.444476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:55.444476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:55.444476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:55.444476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:55.444476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:55.444476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:55.445352 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:55.445352 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:55.445352 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:55.445352 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:55.445352 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:55.445352 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:55.445352 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:55.445352 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:55.445352 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:55.445352 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:55.445352 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:55.445352 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:55.445352 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:55.445352 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:55.445352 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:55.445352 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:55.446121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:55.446121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:55.446121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:55.446121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:55.446121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:55.446121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:55.446121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:55.446121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:55.446121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:55.446121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:55.446121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:28:55.446121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:28:55.446703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269d580-cd97-42cc-841d-e53d342d640e None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:55.446703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269d580-cd97-42cc-841d-e53d342d640e None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:55.446703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269d580-cd97-42cc-841d-e53d342d640e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:55.446864 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 135/273] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:28:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:28:55.451582 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6642bb92-7704-4aa4-99a4-fda39789f262 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:55.463644 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-8269d580-cd97-42cc-841d-e53d342d640e None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:55.464850 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-8269d580-cd97-42cc-841d-e53d342d640e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:55.547425 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 139/274] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:28:55.572000 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269d580-cd97-42cc-841d-e53d342d640e None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:55.583629 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:55.602242 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6642bb92-7704-4aa4-99a4-fda39789f262 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:55.639035 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c716aac-0723-4049-ba6a-4e2e8731ae48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:55.639704 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c716aac-0723-4049-ba6a-4e2e8731ae48 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:55.639984 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c716aac-0723-4049-ba6a-4e2e8731ae48 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:55.640262 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c716aac-0723-4049-ba6a-4e2e8731ae48 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:55.640993 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c716aac-0723-4049-ba6a-4e2e8731ae48 None admin] RBAC: Adding query filter params (name=member) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:55.641296 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c716aac-0723-4049-ba6a-4e2e8731ae48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:55.642954 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c716aac-0723-4049-ba6a-4e2e8731ae48 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:55.652742 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 136/275] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Jun 15 14:28:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:55.657844 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c716aac-0723-4049-ba6a-4e2e8731ae48 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:55.703843 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269d580-cd97-42cc-841d-e53d342d640e None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:55.721933 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:55.752320 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513be75-97c1-4fbd-af56-c406745303c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:55.753122 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513be75-97c1-4fbd-af56-c406745303c0 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:55.753401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513be75-97c1-4fbd-af56-c406745303c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:55.753676 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513be75-97c1-4fbd-af56-c406745303c0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:55.766353 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a513be75-97c1-4fbd-af56-c406745303c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:55.774456 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-a513be75-97c1-4fbd-af56-c406745303c0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:28:55.774456 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:55.774456 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:55.774456 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:55.774456 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:55.774456 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:55.774456 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:55.774456 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:55.774456 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:55.774456 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:55.774456 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:55.774456 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:55.774456 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:55.774456 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:55.774456 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:55.774456 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:55.775381 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:55.775381 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:55.775381 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:55.775381 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:55.775381 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:55.775381 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:55.775381 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:55.775381 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:55.775381 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:55.775381 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:55.775381 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:55.775381 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:55.775381 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:55.775381 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:55.775381 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:55.775381 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:55.776136 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:55.776136 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:55.776136 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:55.776136 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:55.776136 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:55.776136 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:55.776136 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:55.776136 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:55.776136 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:55.776136 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:55.776136 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:28:55.776136 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:28:55.776136 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 140/276] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:28:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:28:55.786279 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513be75-97c1-4fbd-af56-c406745303c0 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:55.803066 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c716aac-0723-4049-ba6a-4e2e8731ae48 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:55.849566 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0e86ad-b61e-4025-aa84-242d5a0ba43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:55.850467 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e86ad-b61e-4025-aa84-242d5a0ba43c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:55.850745 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e86ad-b61e-4025-aa84-242d5a0ba43c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:55.851037 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e86ad-b61e-4025-aa84-242d5a0ba43c None admin] PATH_INFO: `/v3/users/1c20ff4f60154fe186638adcd2524168` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:55.869469 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0e86ad-b61e-4025-aa84-242d5a0ba43c None admin] RBAC: Authorizing `identity:get_user(user_id=1c20ff4f60154fe186638adcd2524168)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:55.871587 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0e86ad-b61e-4025-aa84-242d5a0ba43c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:55.873477 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:55.873792 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 137/277] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:55 2022] GET /identity/v3/users/1c20ff4f60154fe186638adcd2524168 => generated 293 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:55.879324 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0e86ad-b61e-4025-aa84-242d5a0ba43c None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:55.919147 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513be75-97c1-4fbd-af56-c406745303c0 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:55.962784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05390dd-090b-4537-937b-bd5ddc74da79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:55.963492 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05390dd-090b-4537-937b-bd5ddc74da79 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:55.963787 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05390dd-090b-4537-937b-bd5ddc74da79 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:55.964606 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05390dd-090b-4537-937b-bd5ddc74da79 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:55.965361 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05390dd-090b-4537-937b-bd5ddc74da79 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:55.965781 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05390dd-090b-4537-937b-bd5ddc74da79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:55.967203 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05390dd-090b-4537-937b-bd5ddc74da79 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:55.980746 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 141/278] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Jun 15 14:28:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:55.980746 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1bb8f-8c5c-47f6-aaf8-033a671c346e None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:55.980746 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1bb8f-8c5c-47f6-aaf8-033a671c346e None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:55.980746 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1bb8f-8c5c-47f6-aaf8-033a671c346e None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:55.980746 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 142/279] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:55.984316 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1bb8f-8c5c-47f6-aaf8-033a671c346e None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:56.005360 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:56.008780 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0e86ad-b61e-4025-aa84-242d5a0ba43c None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:56.048315 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee377edd-ead1-4ed0-b343-c1046f2f8b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:56.048959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee377edd-ead1-4ed0-b343-c1046f2f8b4e None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:56.049246 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee377edd-ead1-4ed0-b343-c1046f2f8b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:56.049533 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee377edd-ead1-4ed0-b343-c1046f2f8b4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:56.050365 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee377edd-ead1-4ed0-b343-c1046f2f8b4e None admin] RBAC: Adding query filter params (role.id=ce0de9b9c9054826b569db895942ad26, user.id=1c20ff4f60154fe186638adcd2524168, scope.system=all) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:56.050679 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee377edd-ead1-4ed0-b343-c1046f2f8b4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:56.052165 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee377edd-ead1-4ed0-b343-c1046f2f8b4e None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:56.057601 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 138/280] 10.222.0.45 () {56 vars in 1308 bytes} [Wed Jun 15 14:28:55 2022] GET /identity/v3/role_assignments?user.id=1c20ff4f60154fe186638adcd2524168&scope.system=all&role.id=ce0de9b9c9054826b569db895942ad26 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:56.058716 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9d4ca-d114-454f-a91d-e963fafb2ae8 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:56.059068 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9d4ca-d114-454f-a91d-e963fafb2ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:56.059339 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9d4ca-d114-454f-a91d-e963fafb2ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:56.078053 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-c2d9d4ca-d114-454f-a91d-e963fafb2ae8 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:56.079506 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c2d9d4ca-d114-454f-a91d-e963fafb2ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:56.112061 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1bb8f-8c5c-47f6-aaf8-033a671c346e None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:56.145056 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:56.156532 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785809e-599d-4b6e-a5d2-e983ba84c6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:56.157056 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785809e-599d-4b6e-a5d2-e983ba84c6e6 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:56.157344 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785809e-599d-4b6e-a5d2-e983ba84c6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:56.157625 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785809e-599d-4b6e-a5d2-e983ba84c6e6 None admin] PATH_INFO: `/v3/users/56fb5916a497403c8cead2cbebe4d131` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:56.159064 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785809e-599d-4b6e-a5d2-e983ba84c6e6 None admin] RBAC: Authorizing `identity:get_user(user_id=56fb5916a497403c8cead2cbebe4d131)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:56.160523 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785809e-599d-4b6e-a5d2-e983ba84c6e6 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:56.166641 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 143/281] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:55 2022] GET /identity/v3/users/56fb5916a497403c8cead2cbebe4d131 => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:56.167849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785809e-599d-4b6e-a5d2-e983ba84c6e6 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:56.179200 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 139/282] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:56.205701 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d9d4ca-d114-454f-a91d-e963fafb2ae8 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:56.290074 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785809e-599d-4b6e-a5d2-e983ba84c6e6 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:56.313689 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:56.322019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d9d4ca-d114-454f-a91d-e963fafb2ae8 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:56.341484 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dee043-edeb-4cf7-9362-a17a7a5fc3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:56.343362 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dee043-edeb-4cf7-9362-a17a7a5fc3f6 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:56.343660 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dee043-edeb-4cf7-9362-a17a7a5fc3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:56.343943 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dee043-edeb-4cf7-9362-a17a7a5fc3f6 None admin] PATH_INFO: `/v3/projects/59af46f68a07464d897b0c71ece3f2f4` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:56.348275 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dee043-edeb-4cf7-9362-a17a7a5fc3f6 None admin] RBAC: Authorizing `identity:get_project(project_id=59af46f68a07464d897b0c71ece3f2f4)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:56.350637 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dee043-edeb-4cf7-9362-a17a7a5fc3f6 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:56.353883 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 144/283] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:28:56 2022] GET /identity/v3/projects/59af46f68a07464d897b0c71ece3f2f4 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:56.356384 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a91fd-57da-42e8-a5cb-9fbd079d8250 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:56.356678 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a91fd-57da-42e8-a5cb-9fbd079d8250 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:56.356969 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a91fd-57da-42e8-a5cb-9fbd079d8250 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:56.358289 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 145/284] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:56.359899 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a91fd-57da-42e8-a5cb-9fbd079d8250 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:56.371444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d8ed42-fbb8-40f6-ab4c-2436c978e46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:56.372615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8ed42-fbb8-40f6-ab4c-2436c978e46b None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:56.372914 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8ed42-fbb8-40f6-ab4c-2436c978e46b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:56.373196 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8ed42-fbb8-40f6-ab4c-2436c978e46b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:56.374691 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d8ed42-fbb8-40f6-ab4c-2436c978e46b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:56.376185 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d8ed42-fbb8-40f6-ab4c-2436c978e46b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:56.377822 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 140/285] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:28:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:56.380570 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e08aa-b6ec-4c67-932b-5a499bc69858 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:56.380925 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e08aa-b6ec-4c67-932b-5a499bc69858 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:56.381564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e08aa-b6ec-4c67-932b-5a499bc69858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:56.400336 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-0e7e08aa-b6ec-4c67-932b-5a499bc69858 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:56.402188 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-0e7e08aa-b6ec-4c67-932b-5a499bc69858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:56.427694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:56.472594 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a91fd-57da-42e8-a5cb-9fbd079d8250 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:56.475471 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 141/286] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:56.475984 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e08aa-b6ec-4c67-932b-5a499bc69858 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:56.520460 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c0555c-d15e-45bd-b973-0301435ab9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:56.521355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0555c-d15e-45bd-b973-0301435ab9da None admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:56.521641 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0555c-d15e-45bd-b973-0301435ab9da None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:56.521916 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0555c-d15e-45bd-b973-0301435ab9da None admin] PATH_INFO: `/v3/projects/59af46f68a07464d897b0c71ece3f2f4/users/56fb5916a497403c8cead2cbebe4d131/roles/30e8990a9ea24c4398bbe057f41400bb` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:56.523951 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c0555c-d15e-45bd-b973-0301435ab9da None admin] RBAC: Authorizing `identity:create_grant(project_id=59af46f68a07464d897b0c71ece3f2f4, user_id=56fb5916a497403c8cead2cbebe4d131, role_id=30e8990a9ea24c4398bbe057f41400bb)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:56.525369 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c0555c-d15e-45bd-b973-0301435ab9da None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:56.540264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 146/287] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Jun 15 14:28:56 2022] PUT /identity/v3/projects/59af46f68a07464d897b0c71ece3f2f4/users/56fb5916a497403c8cead2cbebe4d131/roles/30e8990a9ea24c4398bbe057f41400bb => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:28:56.541299 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c0555c-d15e-45bd-b973-0301435ab9da None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:56.592039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e08aa-b6ec-4c67-932b-5a499bc69858 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:56.607284 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:56.639880 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de30d2ea-defd-4d39-935e-559a56faa45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:56.640570 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30d2ea-defd-4d39-935e-559a56faa45f None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:56.640854 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30d2ea-defd-4d39-935e-559a56faa45f None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:56.641294 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30d2ea-defd-4d39-935e-559a56faa45f None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:56.642069 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de30d2ea-defd-4d39-935e-559a56faa45f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:56.644406 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de30d2ea-defd-4d39-935e-559a56faa45f None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:56.672874 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 142/288] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Jun 15 14:28:56 2022] POST /identity/v3/users => generated 293 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:28:56.694534 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c0555c-d15e-45bd-b973-0301435ab9da None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:56.724395 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:56.740290 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65946b21-907c-4846-843e-e89eef0ecb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:56.741674 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65946b21-907c-4846-843e-e89eef0ecb61 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:56.741899 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65946b21-907c-4846-843e-e89eef0ecb61 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:56.742156 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65946b21-907c-4846-843e-e89eef0ecb61 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:56.759156 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65946b21-907c-4846-843e-e89eef0ecb61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:56.768241 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-65946b21-907c-4846-843e-e89eef0ecb61 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:28:56.768241 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:56.768241 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:56.768241 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:56.768241 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:56.768241 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:56.768241 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:56.768241 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:56.768241 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:56.768241 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:56.768241 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:56.768241 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:56.768241 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:56.768241 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:56.768241 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:56.768241 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:56.769140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:56.769140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:56.769140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:56.769140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:56.769140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:56.769140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:56.769140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:56.769140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:56.769140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:56.769140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:56.769140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:56.769140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:56.769140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:56.769140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:56.769140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:56.769140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:56.770352 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:56.770352 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:56.770352 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:56.770352 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:56.770352 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:56.770352 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:56.770352 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:56.770352 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:56.770352 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:56.770352 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:56.770352 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:28:56.770352 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:28:56.770352 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 147/289] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:28:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:28:56.775750 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de30d2ea-defd-4d39-935e-559a56faa45f None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:56.891905 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de30d2ea-defd-4d39-935e-559a56faa45f None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:56.927876 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85783fa1-0f5f-4f6b-bd08-a2b2282e2fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:56.928558 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85783fa1-0f5f-4f6b-bd08-a2b2282e2fba None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:56.928844 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85783fa1-0f5f-4f6b-bd08-a2b2282e2fba None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:56.929128 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85783fa1-0f5f-4f6b-bd08-a2b2282e2fba None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:56.929881 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85783fa1-0f5f-4f6b-bd08-a2b2282e2fba None admin] RBAC: Adding query filter params (name=member) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:56.930190 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85783fa1-0f5f-4f6b-bd08-a2b2282e2fba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:56.931702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85783fa1-0f5f-4f6b-bd08-a2b2282e2fba None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:56.941088 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 143/290] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Jun 15 14:28:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:56.946569 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65946b21-907c-4846-843e-e89eef0ecb61 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:57.056514 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65946b21-907c-4846-843e-e89eef0ecb61 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:57.095266 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0199252c-19ed-4326-bd14-e5594718cdf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:57.096143 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199252c-19ed-4326-bd14-e5594718cdf5 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:57.096425 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199252c-19ed-4326-bd14-e5594718cdf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:57.096697 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199252c-19ed-4326-bd14-e5594718cdf5 None admin] PATH_INFO: `/v3/users/a601364422bc416093ce778ef72eacdb` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:57.097893 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0199252c-19ed-4326-bd14-e5594718cdf5 None admin] RBAC: Authorizing `identity:get_user(user_id=a601364422bc416093ce778ef72eacdb)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:57.099391 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0199252c-19ed-4326-bd14-e5594718cdf5 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:57.101098 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 148/291] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:56 2022] GET /identity/v3/users/a601364422bc416093ce778ef72eacdb => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:57.106709 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85783fa1-0f5f-4f6b-bd08-a2b2282e2fba None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:57.121943 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:57.231021 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85783fa1-0f5f-4f6b-bd08-a2b2282e2fba None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:57.272153 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915642fa-34c8-43d1-9f57-76eb9689afcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:57.272816 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915642fa-34c8-43d1-9f57-76eb9689afcd None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:57.273098 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915642fa-34c8-43d1-9f57-76eb9689afcd None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:57.273371 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915642fa-34c8-43d1-9f57-76eb9689afcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:57.371160 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915642fa-34c8-43d1-9f57-76eb9689afcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:57.372627 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915642fa-34c8-43d1-9f57-76eb9689afcd None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:57.389657 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:57.496009 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 144/292] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:28:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:28:57.501757 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0199252c-19ed-4326-bd14-e5594718cdf5 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:57.554311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:57.618935 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0199252c-19ed-4326-bd14-e5594718cdf5 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:57.663796 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26414af3-52e0-432b-b90c-a412a1e4cea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:57.664656 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26414af3-52e0-432b-b90c-a412a1e4cea1 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:57.665083 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26414af3-52e0-432b-b90c-a412a1e4cea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:57.665367 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26414af3-52e0-432b-b90c-a412a1e4cea1 None admin] PATH_INFO: `/v3/projects/59af46f68a07464d897b0c71ece3f2f4` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:57.666563 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26414af3-52e0-432b-b90c-a412a1e4cea1 None admin] RBAC: Authorizing `identity:get_project(project_id=59af46f68a07464d897b0c71ece3f2f4)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:57.668052 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26414af3-52e0-432b-b90c-a412a1e4cea1 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:57.669754 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 149/293] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:28:57 2022] GET /identity/v3/projects/59af46f68a07464d897b0c71ece3f2f4 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:57.675702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915642fa-34c8-43d1-9f57-76eb9689afcd None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:57.834979 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915642fa-34c8-43d1-9f57-76eb9689afcd None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:57.846357 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:57.873294 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad91bd9-2f4e-4656-93ff-f0ddd04fdde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:57.873951 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad91bd9-2f4e-4656-93ff-f0ddd04fdde1 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:57.874231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad91bd9-2f4e-4656-93ff-f0ddd04fdde1 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:57.874506 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad91bd9-2f4e-4656-93ff-f0ddd04fdde1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:57.875373 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad91bd9-2f4e-4656-93ff-f0ddd04fdde1 None admin] RBAC: Adding query filter params (role.id=ce0de9b9c9054826b569db895942ad26, scope.project.id=59af46f68a07464d897b0c71ece3f2f4, user.id=a601364422bc416093ce778ef72eacdb) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:57.875682 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad91bd9-2f4e-4656-93ff-f0ddd04fdde1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:57.877136 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad91bd9-2f4e-4656-93ff-f0ddd04fdde1 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:57.887402 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 145/294] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:28:57 2022] GET /identity/v3/role_assignments?user.id=a601364422bc416093ce778ef72eacdb&scope.project.id=59af46f68a07464d897b0c71ece3f2f4&role.id=ce0de9b9c9054826b569db895942ad26 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:58.519139 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96474a-c34a-4c99-a548-e1727a1c8b4e None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:58.519642 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96474a-c34a-4c99-a548-e1727a1c8b4e None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:58.519716 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96474a-c34a-4c99-a548-e1727a1c8b4e None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:58.521367 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 150/295] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:58.529573 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32130573-edb1-4b71-b019-45602f104b45 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:58.530001 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32130573-edb1-4b71-b019-45602f104b45 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:58.530200 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32130573-edb1-4b71-b019-45602f104b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:58.549073 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-32130573-edb1-4b71-b019-45602f104b45 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:58.550261 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-32130573-edb1-4b71-b019-45602f104b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:58.616951 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 146/296] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:58.639239 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d96474a-c34a-4c99-a548-e1727a1c8b4e None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:58.744756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d96474a-c34a-4c99-a548-e1727a1c8b4e None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:58.762348 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:58.780577 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee381ba8-464c-47c0-8fad-3d5d5cbe19fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:58.781308 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee381ba8-464c-47c0-8fad-3d5d5cbe19fa None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:58.781596 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee381ba8-464c-47c0-8fad-3d5d5cbe19fa None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:58.781871 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee381ba8-464c-47c0-8fad-3d5d5cbe19fa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:58.793134 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee381ba8-464c-47c0-8fad-3d5d5cbe19fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:58.800823 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-ee381ba8-464c-47c0-8fad-3d5d5cbe19fa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:28:58.800823 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:58.800823 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:58.800823 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:58.800823 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:58.800823 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:58.800823 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:58.800823 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:58.800823 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:58.800823 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:58.800823 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:58.800823 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:58.800823 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:58.800823 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:58.800823 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:58.800823 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:58.801669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:58.801669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:58.801669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:58.801669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:58.801669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:58.801669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:58.801669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:58.801669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:58.801669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:58.801669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:58.801669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:58.801669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:58.801669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:58.801669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:58.801669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:58.801669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:58.802433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:58.802433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:58.802433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:58.802433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:58.802433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:58.802433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:58.802433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:58.802433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:58.802433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:58.802433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:58.802433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:28:58.802433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:28:58.802433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 151/297] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:28:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:28:58.808035 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32130573-edb1-4b71-b019-45602f104b45 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:58.911018 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32130573-edb1-4b71-b019-45602f104b45 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:58.914441 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:58.951156 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf8d0f-9cfc-4661-96ef-c6f92f1c2eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:58.951838 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf8d0f-9cfc-4661-96ef-c6f92f1c2eff None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:58.952126 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf8d0f-9cfc-4661-96ef-c6f92f1c2eff None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:58.952407 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf8d0f-9cfc-4661-96ef-c6f92f1c2eff None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:58.953169 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf8d0f-9cfc-4661-96ef-c6f92f1c2eff None admin] RBAC: Adding query filter params (name=member) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:58.953481 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf8d0f-9cfc-4661-96ef-c6f92f1c2eff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:58.954896 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf8d0f-9cfc-4661-96ef-c6f92f1c2eff None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:58.964518 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 147/298] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Jun 15 14:28:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:58.969795 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee381ba8-464c-47c0-8fad-3d5d5cbe19fa None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:58.984294 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de4227-bfc2-4f50-b8b8-6b3753e9a6e4 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:58.984883 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de4227-bfc2-4f50-b8b8-6b3753e9a6e4 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:58.985367 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de4227-bfc2-4f50-b8b8-6b3753e9a6e4 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:58.987325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 148/299] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:28:58.992441 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ac5af-4b1d-4d62-b193-a785b6760741 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:58.992954 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ac5af-4b1d-4d62-b193-a785b6760741 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:58.993441 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ac5af-4b1d-4d62-b193-a785b6760741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:59.011372 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-f55ac5af-4b1d-4d62-b193-a785b6760741 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:59.012908 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f55ac5af-4b1d-4d62-b193-a785b6760741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:59.093094 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee381ba8-464c-47c0-8fad-3d5d5cbe19fa None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:59.093953 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 149/300] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:28:59.116908 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ac5af-4b1d-4d62-b193-a785b6760741 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:59.132473 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a3b64c-98df-4eb2-98f2-8eff26934546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:59.133818 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3b64c-98df-4eb2-98f2-8eff26934546 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:59.134103 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3b64c-98df-4eb2-98f2-8eff26934546 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:59.134380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3b64c-98df-4eb2-98f2-8eff26934546 None admin] PATH_INFO: `/v3/users/1c20ff4f60154fe186638adcd2524168` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:59.136413 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a3b64c-98df-4eb2-98f2-8eff26934546 None admin] RBAC: Authorizing `identity:get_user(user_id=1c20ff4f60154fe186638adcd2524168)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:59.138522 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a3b64c-98df-4eb2-98f2-8eff26934546 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:59.140507 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 152/301] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:58 2022] GET /identity/v3/users/1c20ff4f60154fe186638adcd2524168 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:59.146110 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a3b64c-98df-4eb2-98f2-8eff26934546 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:59.195086 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:59.221794 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:59.228903 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ac5af-4b1d-4d62-b193-a785b6760741 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:59.254231 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a3b64c-98df-4eb2-98f2-8eff26934546 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:59.277348 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da53ec-a321-4ef6-a3c7-030aa133b4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:59.278710 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da53ec-a321-4ef6-a3c7-030aa133b4d5 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:59.279400 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da53ec-a321-4ef6-a3c7-030aa133b4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:59.280266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da53ec-a321-4ef6-a3c7-030aa133b4d5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:59.294423 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00da53ec-a321-4ef6-a3c7-030aa133b4d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:59.299966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ad4eb-b41b-485f-8e75-7c510256a448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:59.301128 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ad4eb-b41b-485f-8e75-7c510256a448 None admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:59.301625 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ad4eb-b41b-485f-8e75-7c510256a448 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:59.302096 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ad4eb-b41b-485f-8e75-7c510256a448 None admin] PATH_INFO: `/v3/system/users/1c20ff4f60154fe186638adcd2524168/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:59.302564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-00da53ec-a321-4ef6-a3c7-030aa133b4d5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:28:59.302564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:59.302564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:59.302564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:59.302564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:59.302564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:59.302564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:59.302564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:59.302564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:59.302564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:59.302564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:59.302564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:59.302564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:59.302564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:59.302564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:59.302564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:59.303418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:59.303418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:59.303418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:59.303418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:59.303418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:59.303418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:59.303418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:59.303418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:59.303418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:59.303418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:59.303418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:59.303418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:59.303418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:59.303418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:59.303418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:59.303418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:59.304223 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:59.304223 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:59.304223 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:59.304223 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:59.304223 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:59.304223 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:59.304223 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:59.304223 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:59.304223 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:59.304223 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:59.304223 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:28:59.304223 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:28:59.306738 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ad4eb-b41b-485f-8e75-7c510256a448 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1c20ff4f60154fe186638adcd2524168, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:59.308531 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ad4eb-b41b-485f-8e75-7c510256a448 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:59.308799 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 150/302] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:28:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:28:59.309646 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da53ec-a321-4ef6-a3c7-030aa133b4d5 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:59.322132 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 153/303] 10.222.0.45 () {58 vars in 1334 bytes} [Wed Jun 15 14:28:59 2022] PUT /identity/v3/system/users/1c20ff4f60154fe186638adcd2524168/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:28:59.416369 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562aff3c-f714-4066-8812-1e77170cbb70 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:59.416647 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562aff3c-f714-4066-8812-1e77170cbb70 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:59.418852 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562aff3c-f714-4066-8812-1e77170cbb70 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:59.418852 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 154/304] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:28:59.424844 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32bc659-0728-4a38-a24c-2d1ac215da15 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:59.425171 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32bc659-0728-4a38-a24c-2d1ac215da15 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:59.425413 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32bc659-0728-4a38-a24c-2d1ac215da15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:59.447891 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-d32bc659-0728-4a38-a24c-2d1ac215da15 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:59.449930 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d32bc659-0728-4a38-a24c-2d1ac215da15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:59.472207 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da53ec-a321-4ef6-a3c7-030aa133b4d5 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:59.518708 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bb3989-1719-4dee-bee1-b0b38b60aacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:59.519415 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb3989-1719-4dee-bee1-b0b38b60aacf None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:59.519695 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb3989-1719-4dee-bee1-b0b38b60aacf None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:59.519972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb3989-1719-4dee-bee1-b0b38b60aacf None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:59.520723 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bb3989-1719-4dee-bee1-b0b38b60aacf None admin] RBAC: Adding query filter params (name=reader) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:59.521028 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bb3989-1719-4dee-bee1-b0b38b60aacf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:59.522444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bb3989-1719-4dee-bee1-b0b38b60aacf None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:59.532212 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 151/305] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Jun 15 14:28:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:59.533681 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1ca36-53f4-47d9-980d-34d54769a99c None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:59.533969 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1ca36-53f4-47d9-980d-34d54769a99c None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:59.534241 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1ca36-53f4-47d9-980d-34d54769a99c None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:59.535522 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 152/306] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:59.537787 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 155/307] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:59.541480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1ca36-53f4-47d9-980d-34d54769a99c None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:59.543498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe6af9-dde7-4f7e-a47d-5f3b77868e27 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:59.543801 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe6af9-dde7-4f7e-a47d-5f3b77868e27 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:59.544063 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe6af9-dde7-4f7e-a47d-5f3b77868e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:59.566187 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-9efe6af9-dde7-4f7e-a47d-5f3b77868e27 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:59.567058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:59.567991 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-9efe6af9-dde7-4f7e-a47d-5f3b77868e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:59.604508 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:59.641072 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 156/308] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:28:59.647212 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe6af9-dde7-4f7e-a47d-5f3b77868e27 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:59.660751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1ca36-53f4-47d9-980d-34d54769a99c None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:59.699058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d8449a-87d6-4a24-9312-b7254c81be00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:59.699867 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8449a-87d6-4a24-9312-b7254c81be00 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:59.700150 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8449a-87d6-4a24-9312-b7254c81be00 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:59.700423 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8449a-87d6-4a24-9312-b7254c81be00 None admin] PATH_INFO: `/v3/users/56fb5916a497403c8cead2cbebe4d131` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:59.701631 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d8449a-87d6-4a24-9312-b7254c81be00 None admin] RBAC: Authorizing `identity:get_user(user_id=56fb5916a497403c8cead2cbebe4d131)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:59.704581 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d8449a-87d6-4a24-9312-b7254c81be00 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:59.704905 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 153/309] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:59 2022] GET /identity/v3/users/56fb5916a497403c8cead2cbebe4d131 => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:59.707927 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d8449a-87d6-4a24-9312-b7254c81be00 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:59.765549 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe6af9-dde7-4f7e-a47d-5f3b77868e27 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:59.795959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:59.804886 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7020b-9c2c-4c91-9e5c-7c7a83346ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:59.806142 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7020b-9c2c-4c91-9e5c-7c7a83346ebc None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:59.806429 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7020b-9c2c-4c91-9e5c-7c7a83346ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:59.806846 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7020b-9c2c-4c91-9e5c-7c7a83346ebc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:59.812965 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d8449a-87d6-4a24-9312-b7254c81be00 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:59.817949 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e7020b-9c2c-4c91-9e5c-7c7a83346ebc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:59.826244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-a6e7020b-9c2c-4c91-9e5c-7c7a83346ebc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:28:59.826244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:59.826244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:59.826244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:59.826244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:59.826244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:59.826244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:59.826244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:59.826244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:59.826244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:59.826244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:59.826244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:59.826244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:59.826244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:59.826244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:59.826244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:59.827340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:59.827340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:59.827340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:59.827340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:59.827340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:59.827340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:59.827340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:59.827340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:59.827340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:59.827340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:59.827340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:59.827340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:59.827340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:59.827340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:59.827340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:59.827340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:59.828167 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:59.828167 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:59.828167 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:59.828167 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:59.828167 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:59.828167 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:59.828167 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:59.828167 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:59.828167 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:59.828167 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:59.828167 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:28:59.828167 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:28:59.828167 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 157/310] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:28:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:28:59.828820 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7020b-9c2c-4c91-9e5c-7c7a83346ebc None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:59.858463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311a77a-674d-4605-a25c-ef190b5cd631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:59.859328 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311a77a-674d-4605-a25c-ef190b5cd631 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:59.859610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311a77a-674d-4605-a25c-ef190b5cd631 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:59.859882 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311a77a-674d-4605-a25c-ef190b5cd631 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:59.860932 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:59.861057 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311a77a-674d-4605-a25c-ef190b5cd631 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:59.862595 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311a77a-674d-4605-a25c-ef190b5cd631 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:59.868913 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 154/311] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:28:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:59.868913 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311a77a-674d-4605-a25c-ef190b5cd631 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:59.941612 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7020b-9c2c-4c91-9e5c-7c7a83346ebc None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:59.972302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311a77a-674d-4605-a25c-ef190b5cd631 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:59.979314 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a63f31-cdaa-4bb4-a6c4-48824f679ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:59.980506 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a63f31-cdaa-4bb4-a6c4-48824f679ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:59.980792 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a63f31-cdaa-4bb4-a6c4-48824f679ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:59.981736 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a63f31-cdaa-4bb4-a6c4-48824f679ea1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:00.014934 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0685738c-8f04-42c9-9f57-31cb22f3d20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:00.015652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685738c-8f04-42c9-9f57-31cb22f3d20b None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:00.015936 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685738c-8f04-42c9-9f57-31cb22f3d20b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:00.016213 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685738c-8f04-42c9-9f57-31cb22f3d20b None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:00.016954 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0685738c-8f04-42c9-9f57-31cb22f3d20b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:00.017267 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0685738c-8f04-42c9-9f57-31cb22f3d20b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:00.018943 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0685738c-8f04-42c9-9f57-31cb22f3d20b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:00.028125 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 155/312] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Jun 15 14:28:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:00.028921 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0685738c-8f04-42c9-9f57-31cb22f3d20b None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:00.047428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:00.060630 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a63f31-cdaa-4bb4-a6c4-48824f679ea1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:00.062105 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a63f31-cdaa-4bb4-a6c4-48824f679ea1 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:00.093448 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:00.132720 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0685738c-8f04-42c9-9f57-31cb22f3d20b None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:00.169865 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e806a95-c028-4648-921e-c92b2816bea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:00.170535 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e806a95-c028-4648-921e-c92b2816bea9 None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:00.170820 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e806a95-c028-4648-921e-c92b2816bea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:00.171127 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e806a95-c028-4648-921e-c92b2816bea9 None admin] PATH_INFO: `/v3/groups` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:00.171818 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e806a95-c028-4648-921e-c92b2816bea9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:00.173290 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e806a95-c028-4648-921e-c92b2816bea9 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:00.173743 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 158/313] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:28:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:00.174646 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a63f31-cdaa-4bb4-a6c4-48824f679ea1 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:00.199040 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 156/314] 10.222.0.45 () {60 vars in 1134 bytes} [Wed Jun 15 14:29:00 2022] POST /identity/v3/groups => generated 229 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:29:00.199040 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e806a95-c028-4648-921e-c92b2816bea9 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:00.301261 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a63f31-cdaa-4bb4-a6c4-48824f679ea1 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:00.341040 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:00.342414 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:00.349730 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4947d-dd6e-4f69-a548-8b74c315b04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:00.350880 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4947d-dd6e-4f69-a548-8b74c315b04d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:00.351328 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4947d-dd6e-4f69-a548-8b74c315b04d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:00.351735 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4947d-dd6e-4f69-a548-8b74c315b04d None admin] PATH_INFO: `/v3/users/693c4da2c47946df9fc7bdd9af6cc0c2` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:00.369672 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d4947d-dd6e-4f69-a548-8b74c315b04d None admin] RBAC: Authorizing `identity:get_user(user_id=693c4da2c47946df9fc7bdd9af6cc0c2)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:00.371410 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e806a95-c028-4648-921e-c92b2816bea9 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:00.371706 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d4947d-dd6e-4f69-a548-8b74c315b04d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:00.373578 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 159/315] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:29:00 2022] GET /identity/v3/users/693c4da2c47946df9fc7bdd9af6cc0c2 => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:00.379942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4947d-dd6e-4f69-a548-8b74c315b04d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:00.416892 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e49ec0-7aa8-4748-aeb8-30585bd48e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:00.417520 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e49ec0-7aa8-4748-aeb8-30585bd48e04 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:00.417751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e49ec0-7aa8-4748-aeb8-30585bd48e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:00.418006 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e49ec0-7aa8-4748-aeb8-30585bd48e04 None admin] PATH_INFO: `/v3/projects/59af46f68a07464d897b0c71ece3f2f4` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:00.419232 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e49ec0-7aa8-4748-aeb8-30585bd48e04 None admin] RBAC: Authorizing `identity:get_project(project_id=59af46f68a07464d897b0c71ece3f2f4)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:00.420683 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e49ec0-7aa8-4748-aeb8-30585bd48e04 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:00.422389 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 157/316] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:00 2022] GET /identity/v3/projects/59af46f68a07464d897b0c71ece3f2f4 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:00.428190 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e49ec0-7aa8-4748-aeb8-30585bd48e04 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:00.505948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4947d-dd6e-4f69-a548-8b74c315b04d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:00.553316 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e49ec0-7aa8-4748-aeb8-30585bd48e04 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:00.553795 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc2f06-2b2b-46fb-bcc6-be1cf310dc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:00.554334 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc2f06-2b2b-46fb-bcc6-be1cf310dc35 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:00.554620 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc2f06-2b2b-46fb-bcc6-be1cf310dc35 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:00.555080 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc2f06-2b2b-46fb-bcc6-be1cf310dc35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:00.556734 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc2f06-2b2b-46fb-bcc6-be1cf310dc35 None admin] RBAC: Adding query filter params (role.id=30e8990a9ea24c4398bbe057f41400bb, user.id=693c4da2c47946df9fc7bdd9af6cc0c2, scope.system=all) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:00.557058 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc2f06-2b2b-46fb-bcc6-be1cf310dc35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:00.558587 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc2f06-2b2b-46fb-bcc6-be1cf310dc35 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:00.567091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 160/317] 10.222.0.45 () {56 vars in 1308 bytes} [Wed Jun 15 14:29:00 2022] GET /identity/v3/role_assignments?user.id=693c4da2c47946df9fc7bdd9af6cc0c2&scope.system=all&role.id=30e8990a9ea24c4398bbe057f41400bb => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:00.597863 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c8371-af6c-451b-98e1-ae2c4b58cac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:00.598550 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c8371-af6c-451b-98e1-ae2c4b58cac4 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:00.598839 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c8371-af6c-451b-98e1-ae2c4b58cac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:00.599155 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c8371-af6c-451b-98e1-ae2c4b58cac4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:00.600134 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c8371-af6c-451b-98e1-ae2c4b58cac4 None admin] RBAC: Adding query filter params (role.id=30e8990a9ea24c4398bbe057f41400bb, scope.project.id=59af46f68a07464d897b0c71ece3f2f4, user.id=56fb5916a497403c8cead2cbebe4d131) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:00.600455 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c8371-af6c-451b-98e1-ae2c4b58cac4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:00.601972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c8371-af6c-451b-98e1-ae2c4b58cac4 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:00.609934 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 158/318] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:29:00 2022] GET /identity/v3/role_assignments?user.id=56fb5916a497403c8cead2cbebe4d131&scope.project.id=59af46f68a07464d897b0c71ece3f2f4&role.id=30e8990a9ea24c4398bbe057f41400bb => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:01.532656 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51145a9d-c652-4ae5-bd95-664368608ab2 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:01.533030 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51145a9d-c652-4ae5-bd95-664368608ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:01.533226 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51145a9d-c652-4ae5-bd95-664368608ab2 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:01.534653 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 161/319] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:01.539773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0bcec-38ef-4c69-a36f-183865301484 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:01.540072 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0bcec-38ef-4c69-a36f-183865301484 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:01.540350 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0bcec-38ef-4c69-a36f-183865301484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:01.556954 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-aee0bcec-38ef-4c69-a36f-183865301484 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:01.558186 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-aee0bcec-38ef-4c69-a36f-183865301484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:01.560443 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:01.630667 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 159/320] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:01.654470 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51145a9d-c652-4ae5-bd95-664368608ab2 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:01.713751 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:01.766974 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51145a9d-c652-4ae5-bd95-664368608ab2 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:01.801132 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb934c-3131-4543-a582-ca6d9dc77d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:01.801928 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb934c-3131-4543-a582-ca6d9dc77d3b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:01.802212 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb934c-3131-4543-a582-ca6d9dc77d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:01.802488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb934c-3131-4543-a582-ca6d9dc77d3b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:01.813060 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb934c-3131-4543-a582-ca6d9dc77d3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:01.820847 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-23eb934c-3131-4543-a582-ca6d9dc77d3b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:01.820847 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:01.820847 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:01.820847 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:01.820847 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:01.820847 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:01.820847 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:01.820847 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:01.820847 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:01.820847 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:01.820847 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:01.820847 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:01.820847 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:01.820847 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:01.820847 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:01.820847 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:01.821665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:01.821665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:01.821665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:01.821665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:01.821665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:01.821665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:01.821665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:01.821665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:01.821665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:01.821665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:01.821665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:01.821665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:01.821665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:01.821665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:01.821665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:01.821665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:01.822462 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:01.822462 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:01.822462 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:01.822462 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:01.822462 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:01.822462 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:01.822462 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:01.822462 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:01.822462 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:01.822462 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:01.822462 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:01.822462 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:01.822462 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 162/321] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:01.828343 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0bcec-38ef-4c69-a36f-183865301484 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:01.927180 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0bcec-38ef-4c69-a36f-183865301484 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:01.960827 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f86584-9289-4562-bc56-4105134e66a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:01.961489 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f86584-9289-4562-bc56-4105134e66a8 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:01.961769 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f86584-9289-4562-bc56-4105134e66a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:01.962036 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f86584-9289-4562-bc56-4105134e66a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:01.962774 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f86584-9289-4562-bc56-4105134e66a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:01.963108 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f86584-9289-4562-bc56-4105134e66a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:01.964862 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f86584-9289-4562-bc56-4105134e66a8 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:01.971911 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:01.976777 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 160/322] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Jun 15 14:29:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:01.982079 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb934c-3131-4543-a582-ca6d9dc77d3b None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:02.082352 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb934c-3131-4543-a582-ca6d9dc77d3b None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:02.094869 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:02.117247 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de61b476-1ee8-4092-a773-30a8df4a7c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:02.118115 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de61b476-1ee8-4092-a773-30a8df4a7c3b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:02.118452 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de61b476-1ee8-4092-a773-30a8df4a7c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:02.118730 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de61b476-1ee8-4092-a773-30a8df4a7c3b None admin] PATH_INFO: `/v3/users/1c20ff4f60154fe186638adcd2524168` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:02.119969 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de61b476-1ee8-4092-a773-30a8df4a7c3b None admin] RBAC: Authorizing `identity:get_user(user_id=1c20ff4f60154fe186638adcd2524168)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:02.121949 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de61b476-1ee8-4092-a773-30a8df4a7c3b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:02.127098 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 163/323] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:29:01 2022] GET /identity/v3/users/1c20ff4f60154fe186638adcd2524168 => generated 293 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:02.129686 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f86584-9289-4562-bc56-4105134e66a8 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:02.232408 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f86584-9289-4562-bc56-4105134e66a8 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:02.265849 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51246fc5-960b-4283-91a7-42a40c0e575f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:02.266505 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51246fc5-960b-4283-91a7-42a40c0e575f None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:02.266790 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51246fc5-960b-4283-91a7-42a40c0e575f None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:02.267092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51246fc5-960b-4283-91a7-42a40c0e575f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:02.268185 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51246fc5-960b-4283-91a7-42a40c0e575f None admin] RBAC: Adding query filter params (role.id=ce0de9b9c9054826b569db895942ad26, user.id=1c20ff4f60154fe186638adcd2524168, scope.system=all) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:02.268497 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51246fc5-960b-4283-91a7-42a40c0e575f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:02.269943 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51246fc5-960b-4283-91a7-42a40c0e575f None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:02.281996 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 161/324] 10.222.0.45 () {56 vars in 1308 bytes} [Wed Jun 15 14:29:02 2022] GET /identity/v3/role_assignments?user.id=1c20ff4f60154fe186638adcd2524168&scope.system=all&role.id=ce0de9b9c9054826b569db895942ad26 => generated 510 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:02.412646 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4c4f8-315d-4b3c-9416-a578c2c59933 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:02.413040 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4c4f8-315d-4b3c-9416-a578c2c59933 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:02.413492 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4c4f8-315d-4b3c-9416-a578c2c59933 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:02.415526 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 164/325] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:02.421646 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9ec42-239b-4eb3-9a46-0c7e5f346705 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:02.422020 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9ec42-239b-4eb3-9a46-0c7e5f346705 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:02.422306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9ec42-239b-4eb3-9a46-0c7e5f346705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:02.440931 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-03f9ec42-239b-4eb3-9a46-0c7e5f346705 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:02.442149 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-03f9ec42-239b-4eb3-9a46-0c7e5f346705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:02.498779 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:02.513345 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 162/326] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:02.536291 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a4c4f8-315d-4b3c-9416-a578c2c59933 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:02.635669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a4c4f8-315d-4b3c-9416-a578c2c59933 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:02.669643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e21f56-9159-4e5e-b9d5-3b5a8f533e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:02.670383 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e21f56-9159-4e5e-b9d5-3b5a8f533e6c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:02.670663 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e21f56-9159-4e5e-b9d5-3b5a8f533e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:02.670936 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e21f56-9159-4e5e-b9d5-3b5a8f533e6c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:02.672225 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e21f56-9159-4e5e-b9d5-3b5a8f533e6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:02.673712 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e21f56-9159-4e5e-b9d5-3b5a8f533e6c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:02.675618 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 165/327] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:02.681219 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9ec42-239b-4eb3-9a46-0c7e5f346705 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:02.731408 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d46005-9290-43cb-bd49-3c0d3824f9d8 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:02.731692 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d46005-9290-43cb-bd49-3c0d3824f9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:02.731967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d46005-9290-43cb-bd49-3c0d3824f9d8 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:02.733339 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 166/328] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:02.739264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb9037-2e7c-4093-9d13-3914af1ce998 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:02.739546 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb9037-2e7c-4093-9d13-3914af1ce998 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:02.739822 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb9037-2e7c-4093-9d13-3914af1ce998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:02.756693 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-9bfb9037-2e7c-4093-9d13-3914af1ce998 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:02.757887 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-9bfb9037-2e7c-4093-9d13-3914af1ce998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:02.784842 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9ec42-239b-4eb3-9a46-0c7e5f346705 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:02.801654 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:02.818021 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751782b-4efc-4bae-a8c6-d93f49795dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:02.818714 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751782b-4efc-4bae-a8c6-d93f49795dba None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:02.819010 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751782b-4efc-4bae-a8c6-d93f49795dba None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:02.819303 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751782b-4efc-4bae-a8c6-d93f49795dba None admin] PATH_INFO: `/v3/groups` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:02.819987 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751782b-4efc-4bae-a8c6-d93f49795dba None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:02.821427 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751782b-4efc-4bae-a8c6-d93f49795dba None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:02.824389 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 167/329] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:29:02.847018 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 163/330] 10.222.0.45 () {60 vars in 1134 bytes} [Wed Jun 15 14:29:02 2022] POST /identity/v3/groups => generated 226 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:29:02.847908 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb9037-2e7c-4093-9d13-3914af1ce998 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:02.960842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb9037-2e7c-4093-9d13-3914af1ce998 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:02.996528 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a49c1-a1ce-413e-933d-ede1251833e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:02.997327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a49c1-a1ce-413e-933d-ede1251833e7 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:02.997609 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a49c1-a1ce-413e-933d-ede1251833e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:02.997885 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a49c1-a1ce-413e-933d-ede1251833e7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:03.008644 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a49c1-a1ce-413e-933d-ede1251833e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:03.016888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-2a6a49c1-a1ce-413e-933d-ede1251833e7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:29:03.016888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:03.016888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:03.016888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:03.016888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:03.016888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:03.016888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:03.016888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:03.016888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:03.016888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:03.016888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:03.016888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:03.016888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:03.016888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:03.016888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:03.016888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:03.017773 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:03.017773 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:03.017773 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:03.017773 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:03.017773 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:03.017773 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:03.017773 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:03.017773 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:03.017773 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:03.017773 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:03.017773 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:03.017773 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:03.017773 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:03.017773 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:03.017773 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:03.017773 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:03.018805 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:03.018805 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:03.018805 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:03.018805 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:03.018805 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:03.018805 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:03.018805 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:03.018805 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:03.018805 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:03.018805 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:03.018805 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:29:03.018805 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:03.018805 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 168/331] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:03.024328 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751782b-4efc-4bae-a8c6-d93f49795dba None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:03.075176 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:03.130299 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751782b-4efc-4bae-a8c6-d93f49795dba None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:03.165560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd6db7-cc96-409b-9eeb-c9f41b810249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:03.166234 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd6db7-cc96-409b-9eeb-c9f41b810249 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:03.166518 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd6db7-cc96-409b-9eeb-c9f41b810249 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:03.166792 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd6db7-cc96-409b-9eeb-c9f41b810249 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:03.167833 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd6db7-cc96-409b-9eeb-c9f41b810249 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:03.168286 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd6db7-cc96-409b-9eeb-c9f41b810249 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:03.169731 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd6db7-cc96-409b-9eeb-c9f41b810249 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:03.178712 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 164/332] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Jun 15 14:29:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:03.184227 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a49c1-a1ce-413e-933d-ede1251833e7 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:03.229565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:03.295420 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a49c1-a1ce-413e-933d-ede1251833e7 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:03.330877 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f363b150-d8bd-4895-a2be-3baf168277c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:03.331794 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363b150-d8bd-4895-a2be-3baf168277c5 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:03.332078 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363b150-d8bd-4895-a2be-3baf168277c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:03.332358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363b150-d8bd-4895-a2be-3baf168277c5 None admin] PATH_INFO: `/v3/users/693c4da2c47946df9fc7bdd9af6cc0c2` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:03.333560 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f363b150-d8bd-4895-a2be-3baf168277c5 None admin] RBAC: Authorizing `identity:get_user(user_id=693c4da2c47946df9fc7bdd9af6cc0c2)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:03.335183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f363b150-d8bd-4895-a2be-3baf168277c5 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:03.337185 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 169/333] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:29:03 2022] GET /identity/v3/users/693c4da2c47946df9fc7bdd9af6cc0c2 => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:03.342435 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd6db7-cc96-409b-9eeb-c9f41b810249 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:03.461577 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd6db7-cc96-409b-9eeb-c9f41b810249 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:03.475908 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:03.506213 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1ef07-c042-45d1-9028-80ca5efe5d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:03.507860 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1ef07-c042-45d1-9028-80ca5efe5d31 None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:03.508356 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1ef07-c042-45d1-9028-80ca5efe5d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:03.508823 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1ef07-c042-45d1-9028-80ca5efe5d31 None admin] PATH_INFO: `/v3/system/users/693c4da2c47946df9fc7bdd9af6cc0c2/roles/30e8990a9ea24c4398bbe057f41400bb` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:03.510569 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1ef07-c042-45d1-9028-80ca5efe5d31 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=693c4da2c47946df9fc7bdd9af6cc0c2, role_id=30e8990a9ea24c4398bbe057f41400bb)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:03.512902 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1ef07-c042-45d1-9028-80ca5efe5d31 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:03.524220 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 165/334] 10.222.0.45 () {58 vars in 1334 bytes} [Wed Jun 15 14:29:03 2022] PUT /identity/v3/system/users/693c4da2c47946df9fc7bdd9af6cc0c2/roles/30e8990a9ea24c4398bbe057f41400bb => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:29:05.472085 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7c95e-fcfc-40a7-9030-365b12db34e2 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:05.472619 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7c95e-fcfc-40a7-9030-365b12db34e2 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:05.472619 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7c95e-fcfc-40a7-9030-365b12db34e2 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:05.474254 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 170/335] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:05.480996 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a446c2a-5f6e-4658-8618-f1bc686a6fd1 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:05.481830 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a446c2a-5f6e-4658-8618-f1bc686a6fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:05.482324 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a446c2a-5f6e-4658-8618-f1bc686a6fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:05.504232 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-2a446c2a-5f6e-4658-8618-f1bc686a6fd1 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:05.505750 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-2a446c2a-5f6e-4658-8618-f1bc686a6fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:05.583938 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 166/336] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:05.610378 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7c95e-fcfc-40a7-9030-365b12db34e2 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:05.730978 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7c95e-fcfc-40a7-9030-365b12db34e2 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:05.741418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:05.772229 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a56dbc-62f6-4725-afc1-7be39859ac3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:05.773389 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a56dbc-62f6-4725-afc1-7be39859ac3c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:05.773928 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a56dbc-62f6-4725-afc1-7be39859ac3c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:05.774419 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a56dbc-62f6-4725-afc1-7be39859ac3c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:05.785810 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a56dbc-62f6-4725-afc1-7be39859ac3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:05.794615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-99a56dbc-62f6-4725-afc1-7be39859ac3c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:05.794615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:05.794615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:05.794615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:05.794615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:05.794615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:05.794615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:05.794615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:05.794615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:05.794615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:05.794615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:05.794615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:05.794615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:05.794615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:05.794615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:05.794615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:05.795510 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:05.795510 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:05.795510 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:05.795510 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:05.795510 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:05.795510 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:05.795510 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:05.795510 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:05.795510 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:05.795510 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:05.795510 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:05.795510 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:05.795510 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:05.795510 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:05.795510 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:05.795510 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:05.796256 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:05.796256 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:05.796256 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:05.796256 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:05.796256 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:05.796256 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:05.796256 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:05.796256 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:05.796256 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:05.796256 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:05.796256 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:05.796256 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:05.798571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 171/337] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:05.804449 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a446c2a-5f6e-4658-8618-f1bc686a6fd1 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:05.927333 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a446c2a-5f6e-4658-8618-f1bc686a6fd1 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:05.930093 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:05.975353 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d9b105-e79a-4a0e-9677-0bd4dbc5d456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:05.976390 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9b105-e79a-4a0e-9677-0bd4dbc5d456 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:05.976947 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9b105-e79a-4a0e-9677-0bd4dbc5d456 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:05.977418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9b105-e79a-4a0e-9677-0bd4dbc5d456 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:05.978376 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d9b105-e79a-4a0e-9677-0bd4dbc5d456 None admin] RBAC: Adding query filter params (name=member) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:05.978876 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d9b105-e79a-4a0e-9677-0bd4dbc5d456 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:05.980530 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d9b105-e79a-4a0e-9677-0bd4dbc5d456 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:05.995692 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 167/338] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Jun 15 14:29:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:06.001453 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a56dbc-62f6-4725-afc1-7be39859ac3c None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:06.121251 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a56dbc-62f6-4725-afc1-7be39859ac3c None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:06.162318 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d0ab9-ae18-4e9f-9210-06f84c3e3975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:06.163519 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d0ab9-ae18-4e9f-9210-06f84c3e3975 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:06.164048 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d0ab9-ae18-4e9f-9210-06f84c3e3975 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:06.164533 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d0ab9-ae18-4e9f-9210-06f84c3e3975 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:06.202776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:06.242236 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714d0ab9-ae18-4e9f-9210-06f84c3e3975 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:06.244031 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714d0ab9-ae18-4e9f-9210-06f84c3e3975 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:06.271401 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e335ba0-2487-4d23-bdc6-aadd10c9c735 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:06.272057 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e335ba0-2487-4d23-bdc6-aadd10c9c735 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:06.272532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e335ba0-2487-4d23-bdc6-aadd10c9c735 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:06.274123 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 168/339] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:29:06.280030 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884350a-a9fb-4fde-8820-b9201227c6a1 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:06.280596 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884350a-a9fb-4fde-8820-b9201227c6a1 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:06.281092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884350a-a9fb-4fde-8820-b9201227c6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:06.300675 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-e884350a-a9fb-4fde-8820-b9201227c6a1 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:06.302164 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e884350a-a9fb-4fde-8820-b9201227c6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:06.404760 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 169/340] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:29:06.406281 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d9a7e-8e81-49e9-9ba5-e189ce3f6123 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:06.406569 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d9a7e-8e81-49e9-9ba5-e189ce3f6123 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:06.406846 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d9a7e-8e81-49e9-9ba5-e189ce3f6123 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:06.408397 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 170/341] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:06.413600 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02feab-54cc-4d79-b3e7-6fae9888a5df None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:06.413874 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02feab-54cc-4d79-b3e7-6fae9888a5df None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:06.414154 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02feab-54cc-4d79-b3e7-6fae9888a5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:06.415588 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 172/342] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:06.422134 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d0ab9-ae18-4e9f-9210-06f84c3e3975 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:06.444471 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-4d02feab-54cc-4d79-b3e7-6fae9888a5df None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:06.445558 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-4d02feab-54cc-4d79-b3e7-6fae9888a5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:06.497361 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:06.554204 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d0ab9-ae18-4e9f-9210-06f84c3e3975 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:06.606521 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 171/343] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:29:06.610110 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02feab-54cc-4d79-b3e7-6fae9888a5df None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:06.630339 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d09d0e-b1c6-4e7b-a4c9-5bfad13bf0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:06.632850 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d09d0e-b1c6-4e7b-a4c9-5bfad13bf0ac None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:06.633608 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d09d0e-b1c6-4e7b-a4c9-5bfad13bf0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:06.634836 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d09d0e-b1c6-4e7b-a4c9-5bfad13bf0ac None admin] PATH_INFO: `/v3/projects/8f586cc781db4f2b96f13e85c4851e5a` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:06.638759 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d09d0e-b1c6-4e7b-a4c9-5bfad13bf0ac None admin] RBAC: Authorizing `identity:get_project(project_id=8f586cc781db4f2b96f13e85c4851e5a)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:06.642651 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d09d0e-b1c6-4e7b-a4c9-5bfad13bf0ac None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:06.652228 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 173/344] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:06 2022] GET /identity/v3/projects/8f586cc781db4f2b96f13e85c4851e5a => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:06.653341 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d09d0e-b1c6-4e7b-a4c9-5bfad13bf0ac None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:06.676762 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:06.786048 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02feab-54cc-4d79-b3e7-6fae9888a5df None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:06.838227 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4577a76-9660-4f5f-b22a-61b89473b30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:06.839107 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4577a76-9660-4f5f-b22a-61b89473b30e None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:06.839394 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4577a76-9660-4f5f-b22a-61b89473b30e None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:06.839666 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4577a76-9660-4f5f-b22a-61b89473b30e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:06.849041 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d09d0e-b1c6-4e7b-a4c9-5bfad13bf0ac None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:06.852981 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4577a76-9660-4f5f-b22a-61b89473b30e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:06.861747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-b4577a76-9660-4f5f-b22a-61b89473b30e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:06.861747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:06.861747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:06.861747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:06.861747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:06.861747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:06.861747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:06.861747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:06.861747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:06.861747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:06.861747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:06.861747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:06.861747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:06.861747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:06.861747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:06.861747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:06.862617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:06.862617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:06.862617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:06.862617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:06.862617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:06.862617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:06.862617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:06.862617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:06.862617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:06.862617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:06.862617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:06.862617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:06.862617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:06.862617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:06.862617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:06.862617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:06.863433 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:06.863433 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:06.863433 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:06.863433 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:06.863433 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:06.863433 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:06.863433 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:06.863433 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:06.863433 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:06.863433 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:06.863433 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:06.863433 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:06.863433 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 172/345] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Jun 15 14:29:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:06.864084 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4577a76-9660-4f5f-b22a-61b89473b30e None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:06.892773 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d99a0c-c61b-4be6-bb9d-934b845aebf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:06.893849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d99a0c-c61b-4be6-bb9d-934b845aebf1 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:06.894137 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d99a0c-c61b-4be6-bb9d-934b845aebf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:06.894416 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d99a0c-c61b-4be6-bb9d-934b845aebf1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:06.898664 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:06.909706 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d99a0c-c61b-4be6-bb9d-934b845aebf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:06.919975 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-18d99a0c-c61b-4be6-bb9d-934b845aebf1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:06.919975 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:06.919975 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:06.919975 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:06.919975 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:06.919975 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:06.919975 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:06.919975 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:06.919975 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:06.919975 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:06.919975 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:06.919975 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:06.919975 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:06.919975 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:06.919975 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:06.919975 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:06.920876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:06.920876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:06.920876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:06.920876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:06.920876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:06.920876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:06.920876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:06.920876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:06.920876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:06.920876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:06.920876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:06.920876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:06.920876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:06.920876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:06.920876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:06.920876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:06.921727 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:06.921727 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:06.921727 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:06.921727 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:06.921727 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:06.921727 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:06.921727 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:06.921727 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:06.921727 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:06.921727 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:06.921727 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:06.921727 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:06.921727 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 174/346] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:06.923403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04671d-0d6c-4522-9b73-c9e3418d255a None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:06.923403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04671d-0d6c-4522-9b73-c9e3418d255a None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:06.923403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04671d-0d6c-4522-9b73-c9e3418d255a None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:06.924565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 175/347] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:06.925494 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd01a8c-a814-40f6-870a-976a69a1076c None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:06.925785 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd01a8c-a814-40f6-870a-976a69a1076c None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:06.926062 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd01a8c-a814-40f6-870a-976a69a1076c None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:06.931600 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 176/348] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:06.932656 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd01a8c-a814-40f6-870a-976a69a1076c None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:06.997083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:07.015022 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4577a76-9660-4f5f-b22a-61b89473b30e None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:07.064647 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd01a8c-a814-40f6-870a-976a69a1076c None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:07.069756 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514551f7-13bb-49a1-bc15-9b427052625b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:07.070619 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514551f7-13bb-49a1-bc15-9b427052625b None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:07.070901 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514551f7-13bb-49a1-bc15-9b427052625b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:07.071215 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514551f7-13bb-49a1-bc15-9b427052625b None admin] PATH_INFO: `/v3/groups/42a71e0d44eb4e3b8d35e0c3b45e9bc2` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:07.078205 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514551f7-13bb-49a1-bc15-9b427052625b None admin] RBAC: Authorizing `identity:get_group(group_id=42a71e0d44eb4e3b8d35e0c3b45e9bc2)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:07.079703 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514551f7-13bb-49a1-bc15-9b427052625b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:07.081932 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 173/349] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Jun 15 14:29:06 2022] GET /identity/v3/groups/42a71e0d44eb4e3b8d35e0c3b45e9bc2 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:07.083014 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514551f7-13bb-49a1-bc15-9b427052625b None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:07.116081 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455b100-79f3-418a-91fe-eb94111e54d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:07.117166 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455b100-79f3-418a-91fe-eb94111e54d2 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:07.117686 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455b100-79f3-418a-91fe-eb94111e54d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:07.118158 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455b100-79f3-418a-91fe-eb94111e54d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:07.119382 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5455b100-79f3-418a-91fe-eb94111e54d2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:07.120124 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5455b100-79f3-418a-91fe-eb94111e54d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:07.122452 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5455b100-79f3-418a-91fe-eb94111e54d2 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:07.133059 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 177/350] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:07.134267 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755209d-8700-4a28-a880-48258fbabc54 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:07.134556 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755209d-8700-4a28-a880-48258fbabc54 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:07.134862 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755209d-8700-4a28-a880-48258fbabc54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:07.153698 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-f755209d-8700-4a28-a880-48258fbabc54 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:07.154872 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f755209d-8700-4a28-a880-48258fbabc54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:07.205528 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514551f7-13bb-49a1-bc15-9b427052625b None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:07.226953 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:07.247583 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 178/351] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:07.248892 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe691ca6-6edb-4168-8a0a-a5316bbd2026 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:07.249180 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe691ca6-6edb-4168-8a0a-a5316bbd2026 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:07.249456 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe691ca6-6edb-4168-8a0a-a5316bbd2026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:07.255455 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8af19-426a-4ed7-b5dd-2271d2b48b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:07.256559 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8af19-426a-4ed7-b5dd-2271d2b48b8d None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:07.256860 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8af19-426a-4ed7-b5dd-2271d2b48b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:07.257140 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8af19-426a-4ed7-b5dd-2271d2b48b8d None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:07.257888 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8af19-426a-4ed7-b5dd-2271d2b48b8d None admin] RBAC: Adding query filter params (name=member) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:07.258200 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8af19-426a-4ed7-b5dd-2271d2b48b8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:07.260794 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8af19-426a-4ed7-b5dd-2271d2b48b8d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:07.272301 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-fe691ca6-6edb-4168-8a0a-a5316bbd2026 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:07.273808 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-fe691ca6-6edb-4168-8a0a-a5316bbd2026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:07.275167 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 174/352] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Jun 15 14:29:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:07.276129 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8af19-426a-4ed7-b5dd-2271d2b48b8d None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:07.292837 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:07.368433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 179/353] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:07.369086 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe691ca6-6edb-4168-8a0a-a5316bbd2026 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:07.424270 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8af19-426a-4ed7-b5dd-2271d2b48b8d None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:07.466635 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09497eb-0b49-4591-aa94-0d8bed2dbeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:07.466635 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09497eb-0b49-4591-aa94-0d8bed2dbeb0 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:07.466635 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09497eb-0b49-4591-aa94-0d8bed2dbeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:07.466968 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09497eb-0b49-4591-aa94-0d8bed2dbeb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:07.467780 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09497eb-0b49-4591-aa94-0d8bed2dbeb0 None admin] RBAC: Adding query filter params (group.id=42a71e0d44eb4e3b8d35e0c3b45e9bc2, role.id=ce0de9b9c9054826b569db895942ad26, scope.project.id=8f586cc781db4f2b96f13e85c4851e5a) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:07.468091 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09497eb-0b49-4591-aa94-0d8bed2dbeb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:07.469534 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09497eb-0b49-4591-aa94-0d8bed2dbeb0 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:07.475303 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 175/354] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Jun 15 14:29:07 2022] GET /identity/v3/role_assignments?group.id=42a71e0d44eb4e3b8d35e0c3b45e9bc2&scope.project.id=8f586cc781db4f2b96f13e85c4851e5a&role.id=ce0de9b9c9054826b569db895942ad26 => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:07.476187 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09497eb-0b49-4591-aa94-0d8bed2dbeb0 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:07.508262 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe691ca6-6edb-4168-8a0a-a5316bbd2026 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:07.536842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:07.568795 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9008d99-3811-4cc0-8f0d-35282141230c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:07.569468 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9008d99-3811-4cc0-8f0d-35282141230c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:07.569751 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9008d99-3811-4cc0-8f0d-35282141230c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:07.570026 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9008d99-3811-4cc0-8f0d-35282141230c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:07.591915 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:07.621331 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09497eb-0b49-4591-aa94-0d8bed2dbeb0 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:07.670332 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9008d99-3811-4cc0-8f0d-35282141230c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:07.672018 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9008d99-3811-4cc0-8f0d-35282141230c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:07.678179 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bacc85-cc3e-4bdc-8761-97b721a893af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:07.679020 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bacc85-cc3e-4bdc-8761-97b721a893af None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:07.679313 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bacc85-cc3e-4bdc-8761-97b721a893af None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:07.679584 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bacc85-cc3e-4bdc-8761-97b721a893af None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:07.700652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bacc85-cc3e-4bdc-8761-97b721a893af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:07.708996 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-69bacc85-cc3e-4bdc-8761-97b721a893af None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:07.708996 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:07.708996 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:07.708996 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:07.708996 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:07.708996 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:07.708996 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:07.708996 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:07.708996 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:07.708996 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:07.708996 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:07.708996 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:07.708996 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:07.708996 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:07.708996 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:07.708996 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:07.709914 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:07.709914 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:07.709914 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:07.709914 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:07.709914 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:07.709914 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:07.709914 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:07.709914 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:07.709914 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:07.709914 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:07.709914 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:07.709914 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:07.709914 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:07.709914 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:07.709914 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:07.709914 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:07.710709 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:07.710709 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:07.710709 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:07.710709 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:07.710709 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:07.710709 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:07.710709 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:07.710709 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:07.710709 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:07.710709 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:07.710709 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:07.710709 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:07.715137 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 176/355] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Jun 15 14:29:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:07.716573 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bacc85-cc3e-4bdc-8761-97b721a893af None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:07.800947 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 180/356] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:07.801805 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9008d99-3811-4cc0-8f0d-35282141230c None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:07.826876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:07.848676 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bacc85-cc3e-4bdc-8761-97b721a893af None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:07.893968 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce4131-d6dd-471f-b9e3-94e912eb6c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:07.894564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce4131-d6dd-471f-b9e3-94e912eb6c0c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:07.894837 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce4131-d6dd-471f-b9e3-94e912eb6c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:07.895154 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce4131-d6dd-471f-b9e3-94e912eb6c0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:07.901192 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:07.938124 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9008d99-3811-4cc0-8f0d-35282141230c None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:07.981644 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1692b4af-abc7-48e9-96f7-498ffecd0dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:07.982439 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692b4af-abc7-48e9-96f7-498ffecd0dee None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:07.982721 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692b4af-abc7-48e9-96f7-498ffecd0dee None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:07.983052 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692b4af-abc7-48e9-96f7-498ffecd0dee None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:07.996975 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1692b4af-abc7-48e9-96f7-498ffecd0dee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:08.000105 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce4131-d6dd-471f-b9e3-94e912eb6c0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:08.001600 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce4131-d6dd-471f-b9e3-94e912eb6c0c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:08.006183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-1692b4af-abc7-48e9-96f7-498ffecd0dee None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:29:08.006183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:08.006183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:08.006183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:08.006183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:08.006183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:08.006183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:08.006183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:08.006183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:08.006183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:08.006183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:08.006183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:08.006183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:08.006183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:08.006183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:08.006183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:08.007141 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:08.007141 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:08.007141 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:08.007141 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:08.007141 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:08.007141 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:08.007141 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:08.007141 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:08.007141 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:08.007141 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:08.007141 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:08.007141 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:08.007141 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:08.007141 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:08.007141 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:08.007141 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:08.007995 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:08.007995 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:08.007995 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:08.007995 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:08.007995 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:08.007995 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:08.007995 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:08.007995 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:08.007995 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:08.007995 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:08.007995 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:29:08.007995 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:08.009098 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 181/357] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:08.010136 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12a0f2-1255-410d-94ca-d71b7aa29c86 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:08.010417 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12a0f2-1255-410d-94ca-d71b7aa29c86 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:08.010760 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12a0f2-1255-410d-94ca-d71b7aa29c86 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:08.013046 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 182/358] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:08.014705 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12a0f2-1255-410d-94ca-d71b7aa29c86 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:08.119743 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:08.137915 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 177/359] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:08.139160 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce4131-d6dd-471f-b9e3-94e912eb6c0c None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:08.161775 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12a0f2-1255-410d-94ca-d71b7aa29c86 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:08.180642 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:08.204599 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb839bd6-de59-40fd-8c45-593dcb3551c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:08.205277 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb839bd6-de59-40fd-8c45-593dcb3551c1 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:08.205562 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb839bd6-de59-40fd-8c45-593dcb3551c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:08.205927 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb839bd6-de59-40fd-8c45-593dcb3551c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:08.206650 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb839bd6-de59-40fd-8c45-593dcb3551c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:08.206961 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb839bd6-de59-40fd-8c45-593dcb3551c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:08.209550 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb839bd6-de59-40fd-8c45-593dcb3551c1 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:08.219148 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 183/360] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:08.219999 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb839bd6-de59-40fd-8c45-593dcb3551c1 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:08.263834 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce4131-d6dd-471f-b9e3-94e912eb6c0c None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:08.311892 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1092d93a-0ece-40b5-88e0-548e798f4442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:08.312614 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092d93a-0ece-40b5-88e0-548e798f4442 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:08.312899 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092d93a-0ece-40b5-88e0-548e798f4442 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:08.313179 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092d93a-0ece-40b5-88e0-548e798f4442 None admin] PATH_INFO: `/v3/projects/8f586cc781db4f2b96f13e85c4851e5a` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:08.314419 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1092d93a-0ece-40b5-88e0-548e798f4442 None admin] RBAC: Authorizing `identity:get_project(project_id=8f586cc781db4f2b96f13e85c4851e5a)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:08.316057 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1092d93a-0ece-40b5-88e0-548e798f4442 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:08.317907 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 178/361] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:08 2022] GET /identity/v3/projects/8f586cc781db4f2b96f13e85c4851e5a => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:08.319181 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4fab6-a9cc-4d5e-a50a-e37625b64105 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:08.319475 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4fab6-a9cc-4d5e-a50a-e37625b64105 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:08.319750 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4fab6-a9cc-4d5e-a50a-e37625b64105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:08.336036 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-ada4fab6-a9cc-4d5e-a50a-e37625b64105 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:08.337244 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ada4fab6-a9cc-4d5e-a50a-e37625b64105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:08.358352 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb839bd6-de59-40fd-8c45-593dcb3551c1 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:08.402282 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ecc4b-04d1-40da-b223-c0ad55bd9cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:08.403235 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ecc4b-04d1-40da-b223-c0ad55bd9cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:08.403686 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ecc4b-04d1-40da-b223-c0ad55bd9cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:08.404113 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ecc4b-04d1-40da-b223-c0ad55bd9cc4 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:08.405005 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ecc4b-04d1-40da-b223-c0ad55bd9cc4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:08.405469 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ecc4b-04d1-40da-b223-c0ad55bd9cc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:08.407029 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ecc4b-04d1-40da-b223-c0ad55bd9cc4 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:08.412938 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 179/362] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:08.414381 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:08.415080 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada4fab6-a9cc-4d5e-a50a-e37625b64105 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:08.423660 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 184/363] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Jun 15 14:29:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:08.424758 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ecc4b-04d1-40da-b223-c0ad55bd9cc4 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:08.458343 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:08.539499 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada4fab6-a9cc-4d5e-a50a-e37625b64105 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:08.559663 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ecc4b-04d1-40da-b223-c0ad55bd9cc4 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:08.583510 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8bd187-1086-4c1b-971a-bf0dd23b5a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:08.584805 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8bd187-1086-4c1b-971a-bf0dd23b5a23 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:08.585315 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8bd187-1086-4c1b-971a-bf0dd23b5a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:08.585793 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8bd187-1086-4c1b-971a-bf0dd23b5a23 None admin] PATH_INFO: `/v3/projects/59af46f68a07464d897b0c71ece3f2f4` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:08.587362 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8bd187-1086-4c1b-971a-bf0dd23b5a23 None admin] RBAC: Authorizing `identity:get_project(project_id=59af46f68a07464d897b0c71ece3f2f4)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:08.589652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8bd187-1086-4c1b-971a-bf0dd23b5a23 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:08.592970 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 180/364] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:08 2022] GET /identity/v3/projects/59af46f68a07464d897b0c71ece3f2f4 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:08.593953 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8bd187-1086-4c1b-971a-bf0dd23b5a23 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:08.609219 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b5ca7-5f7b-4036-9fac-874051fda9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:08.609884 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b5ca7-5f7b-4036-9fac-874051fda9eb None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:08.610169 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b5ca7-5f7b-4036-9fac-874051fda9eb None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:08.610448 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b5ca7-5f7b-4036-9fac-874051fda9eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:08.699520 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:08.704316 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881b5ca7-5f7b-4036-9fac-874051fda9eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:08.706254 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881b5ca7-5f7b-4036-9fac-874051fda9eb None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:08.731440 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8bd187-1086-4c1b-971a-bf0dd23b5a23 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:08.734126 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:08.777165 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1bb8f4-9ca8-40c0-9b26-035a069eaea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:08.778219 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1bb8f4-9ca8-40c0-9b26-035a069eaea0 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:08.778330 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1bb8f4-9ca8-40c0-9b26-035a069eaea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:08.778680 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1bb8f4-9ca8-40c0-9b26-035a069eaea0 None admin] PATH_INFO: `/v3/groups/42a71e0d44eb4e3b8d35e0c3b45e9bc2` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:08.779926 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1bb8f4-9ca8-40c0-9b26-035a069eaea0 None admin] RBAC: Authorizing `identity:get_group(group_id=42a71e0d44eb4e3b8d35e0c3b45e9bc2)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:08.781370 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1bb8f4-9ca8-40c0-9b26-035a069eaea0 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:08.783438 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 181/365] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Jun 15 14:29:08 2022] GET /identity/v3/groups/42a71e0d44eb4e3b8d35e0c3b45e9bc2 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:08.784432 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1bb8f4-9ca8-40c0-9b26-035a069eaea0 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:08.839165 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 185/366] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:08.840023 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b5ca7-5f7b-4036-9fac-874051fda9eb None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:08.908505 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1bb8f4-9ca8-40c0-9b26-035a069eaea0 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:08.943549 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:08.945450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb05a74-7707-49a3-bad1-600cf6a9c2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:08.946325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb05a74-7707-49a3-bad1-600cf6a9c2dd None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:08.946617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb05a74-7707-49a3-bad1-600cf6a9c2dd None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:08.946728 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b5ca7-5f7b-4036-9fac-874051fda9eb None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:08.946887 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb05a74-7707-49a3-bad1-600cf6a9c2dd None admin] PATH_INFO: `/v3/users/693c4da2c47946df9fc7bdd9af6cc0c2` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:08.948167 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb05a74-7707-49a3-bad1-600cf6a9c2dd None admin] RBAC: Authorizing `identity:get_user(user_id=693c4da2c47946df9fc7bdd9af6cc0c2)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:08.950043 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb05a74-7707-49a3-bad1-600cf6a9c2dd None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:08.951502 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 182/367] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:29:08 2022] GET /identity/v3/users/693c4da2c47946df9fc7bdd9af6cc0c2 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:08.952069 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb05a74-7707-49a3-bad1-600cf6a9c2dd None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:08.988965 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174a0af-b59f-4b69-9b07-565d9d75f46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:08.989757 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174a0af-b59f-4b69-9b07-565d9d75f46f None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:08.990045 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174a0af-b59f-4b69-9b07-565d9d75f46f None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:08.990323 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174a0af-b59f-4b69-9b07-565d9d75f46f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:08.993305 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:09.001424 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2174a0af-b59f-4b69-9b07-565d9d75f46f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:09.009099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-2174a0af-b59f-4b69-9b07-565d9d75f46f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:29:09.009099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:09.009099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:09.009099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:09.009099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:09.009099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:09.009099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:09.009099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:09.009099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:09.009099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:09.009099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:09.009099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:09.009099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:09.009099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:09.009099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:09.009099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:09.009942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:09.009942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:09.009942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:09.009942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:09.009942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:09.009942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:09.009942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:09.009942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:09.009942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:09.009942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:09.009942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:09.009942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:09.009942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:09.009942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:09.009942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:09.009942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:09.010703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:09.010703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:09.010703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:09.010703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:09.010703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:09.010703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:09.010703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:09.010703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:09.010703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:09.010703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:09.010703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:29:09.010703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:09.010703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 186/368] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:29:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:09.011999 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174a0af-b59f-4b69-9b07-565d9d75f46f None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:09.059531 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb05a74-7707-49a3-bad1-600cf6a9c2dd None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:09.095391 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab07672e-7c79-4169-9409-3c18cc08a7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:09.096346 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07672e-7c79-4169-9409-3c18cc08a7ec None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:09.096634 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07672e-7c79-4169-9409-3c18cc08a7ec None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:09.097042 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07672e-7c79-4169-9409-3c18cc08a7ec None admin] PATH_INFO: `/v3/groups/42a71e0d44eb4e3b8d35e0c3b45e9bc2` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:09.098308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab07672e-7c79-4169-9409-3c18cc08a7ec None admin] RBAC: Authorizing `identity:get_group(group_id=42a71e0d44eb4e3b8d35e0c3b45e9bc2)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:09.099799 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab07672e-7c79-4169-9409-3c18cc08a7ec None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:09.107350 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 183/369] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Jun 15 14:29:08 2022] GET /identity/v3/groups/42a71e0d44eb4e3b8d35e0c3b45e9bc2 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:09.107932 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab07672e-7c79-4169-9409-3c18cc08a7ec None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:09.117497 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174a0af-b59f-4b69-9b07-565d9d75f46f None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:09.152448 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086578fa-e255-4c99-837d-63bb60e9ba91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:09.153097 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086578fa-e255-4c99-837d-63bb60e9ba91 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:09.153454 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086578fa-e255-4c99-837d-63bb60e9ba91 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:09.153736 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086578fa-e255-4c99-837d-63bb60e9ba91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:09.154562 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086578fa-e255-4c99-837d-63bb60e9ba91 None admin] RBAC: Adding query filter params (group.id=42a71e0d44eb4e3b8d35e0c3b45e9bc2, role.id=6b6353b49c6a41b1a157f8997fc52d9a, scope.project.id=8f586cc781db4f2b96f13e85c4851e5a) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:09.154873 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086578fa-e255-4c99-837d-63bb60e9ba91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:09.156341 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086578fa-e255-4c99-837d-63bb60e9ba91 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:09.171044 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 187/370] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Jun 15 14:29:09 2022] GET /identity/v3/role_assignments?group.id=42a71e0d44eb4e3b8d35e0c3b45e9bc2&scope.project.id=8f586cc781db4f2b96f13e85c4851e5a&role.id=6b6353b49c6a41b1a157f8997fc52d9a => generated 260 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:09.171044 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086578fa-e255-4c99-837d-63bb60e9ba91 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:09.206428 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:09.224712 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab07672e-7c79-4169-9409-3c18cc08a7ec None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:09.236024 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:09.270016 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac2c603-7809-4d13-b671-f1de230a9138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:09.271085 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac2c603-7809-4d13-b671-f1de230a9138 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:09.271597 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac2c603-7809-4d13-b671-f1de230a9138 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:09.271770 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac2c603-7809-4d13-b671-f1de230a9138 None admin] PATH_INFO: `/v3/projects/59af46f68a07464d897b0c71ece3f2f4` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:09.273211 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac2c603-7809-4d13-b671-f1de230a9138 None admin] RBAC: Authorizing `identity:get_project(project_id=59af46f68a07464d897b0c71ece3f2f4)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:09.274801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac2c603-7809-4d13-b671-f1de230a9138 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:09.277672 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 184/371] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:09 2022] GET /identity/v3/projects/59af46f68a07464d897b0c71ece3f2f4 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:09.282944 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac2c603-7809-4d13-b671-f1de230a9138 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:09.335300 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086578fa-e255-4c99-837d-63bb60e9ba91 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:09.377422 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d54fd-6e69-45af-9067-786b57a9a244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:09.378426 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d54fd-6e69-45af-9067-786b57a9a244 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:09.378956 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d54fd-6e69-45af-9067-786b57a9a244 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:09.379442 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d54fd-6e69-45af-9067-786b57a9a244 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:09.380564 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d54fd-6e69-45af-9067-786b57a9a244 None admin] RBAC: Adding query filter params (role.id=30e8990a9ea24c4398bbe057f41400bb, user.id=693c4da2c47946df9fc7bdd9af6cc0c2, scope.system=all) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:09.381099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d54fd-6e69-45af-9067-786b57a9a244 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:09.382777 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d54fd-6e69-45af-9067-786b57a9a244 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:09.396909 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 188/372] 10.222.0.45 () {56 vars in 1308 bytes} [Wed Jun 15 14:29:09 2022] GET /identity/v3/role_assignments?user.id=693c4da2c47946df9fc7bdd9af6cc0c2&scope.system=all&role.id=30e8990a9ea24c4398bbe057f41400bb => generated 510 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:09.398084 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d54fd-6e69-45af-9067-786b57a9a244 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:09.400373 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac2c603-7809-4d13-b671-f1de230a9138 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:09.482966 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67991ee-8309-4c8a-93c8-98f64b354e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:09.483972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67991ee-8309-4c8a-93c8-98f64b354e7e None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:09.484440 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67991ee-8309-4c8a-93c8-98f64b354e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:09.484888 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67991ee-8309-4c8a-93c8-98f64b354e7e None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:09.485816 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67991ee-8309-4c8a-93c8-98f64b354e7e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:09.486290 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67991ee-8309-4c8a-93c8-98f64b354e7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:09.489099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67991ee-8309-4c8a-93c8-98f64b354e7e None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:09.507377 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 185/373] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:09.509444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67991ee-8309-4c8a-93c8-98f64b354e7e None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:09.563380 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:09.572366 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:09.615026 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d54fd-6e69-45af-9067-786b57a9a244 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:09.650285 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67991ee-8309-4c8a-93c8-98f64b354e7e None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:09.660015 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc5ecd3-8cee-4e38-bca9-24ea326cd36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:09.661164 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc5ecd3-8cee-4e38-bca9-24ea326cd36d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:09.661640 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc5ecd3-8cee-4e38-bca9-24ea326cd36d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:09.662121 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc5ecd3-8cee-4e38-bca9-24ea326cd36d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:09.663366 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc5ecd3-8cee-4e38-bca9-24ea326cd36d None admin] RBAC: Adding query filter params (group.id=42a71e0d44eb4e3b8d35e0c3b45e9bc2, role.id=ce0de9b9c9054826b569db895942ad26, scope.project.id=59af46f68a07464d897b0c71ece3f2f4) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:09.663925 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc5ecd3-8cee-4e38-bca9-24ea326cd36d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:09.665739 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc5ecd3-8cee-4e38-bca9-24ea326cd36d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:09.673405 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 189/374] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Jun 15 14:29:09 2022] GET /identity/v3/role_assignments?group.id=42a71e0d44eb4e3b8d35e0c3b45e9bc2&scope.project.id=59af46f68a07464d897b0c71ece3f2f4&role.id=ce0de9b9c9054826b569db895942ad26 => generated 260 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:09.674921 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc5ecd3-8cee-4e38-bca9-24ea326cd36d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:09.717501 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56bf47-f64d-447d-881b-8b6e532380ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:09.718441 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56bf47-f64d-447d-881b-8b6e532380ad None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:09.718730 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56bf47-f64d-447d-881b-8b6e532380ad None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:09.719083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56bf47-f64d-447d-881b-8b6e532380ad None admin] PATH_INFO: `/v3/groups/42a71e0d44eb4e3b8d35e0c3b45e9bc2` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:09.721571 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c56bf47-f64d-447d-881b-8b6e532380ad None admin] RBAC: Authorizing `identity:get_group(group_id=42a71e0d44eb4e3b8d35e0c3b45e9bc2)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:09.723110 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c56bf47-f64d-447d-881b-8b6e532380ad None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:09.725357 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 186/375] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Jun 15 14:29:09 2022] GET /identity/v3/groups/42a71e0d44eb4e3b8d35e0c3b45e9bc2 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:09.731352 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a21107-453e-4fdc-b28f-7c061b3deb52 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:09.731548 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a21107-453e-4fdc-b28f-7c061b3deb52 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:09.732014 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a21107-453e-4fdc-b28f-7c061b3deb52 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:09.736396 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 187/376] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:09 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:09.736708 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a21107-453e-4fdc-b28f-7c061b3deb52 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:09.821311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc5ecd3-8cee-4e38-bca9-24ea326cd36d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:09.867817 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690bbc0f-1ebf-4ed6-a211-d1c9c2e9521c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:09.868808 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690bbc0f-1ebf-4ed6-a211-d1c9c2e9521c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:09.869185 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690bbc0f-1ebf-4ed6-a211-d1c9c2e9521c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:09.869558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690bbc0f-1ebf-4ed6-a211-d1c9c2e9521c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:09.876970 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a21107-453e-4fdc-b28f-7c061b3deb52 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:09.880863 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:09.895378 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:09.935196 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39cad3-2cdc-4a8a-92ab-90e324d42b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:09.935866 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39cad3-2cdc-4a8a-92ab-90e324d42b20 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:09.936205 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39cad3-2cdc-4a8a-92ab-90e324d42b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:09.936507 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39cad3-2cdc-4a8a-92ab-90e324d42b20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:09.938245 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa39cad3-2cdc-4a8a-92ab-90e324d42b20 None admin] RBAC: Adding query filter params (group.id=42a71e0d44eb4e3b8d35e0c3b45e9bc2, role.id=6b6353b49c6a41b1a157f8997fc52d9a, scope.project.id=59af46f68a07464d897b0c71ece3f2f4) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:09.939537 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa39cad3-2cdc-4a8a-92ab-90e324d42b20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:09.941070 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa39cad3-2cdc-4a8a-92ab-90e324d42b20 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:09.952858 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 188/377] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Jun 15 14:29:09 2022] GET /identity/v3/role_assignments?group.id=42a71e0d44eb4e3b8d35e0c3b45e9bc2&scope.project.id=59af46f68a07464d897b0c71ece3f2f4&role.id=6b6353b49c6a41b1a157f8997fc52d9a => generated 260 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:09.952858 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f947cd3-d400-40f8-9da0-a07b34cf1d89 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:09.952858 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f947cd3-d400-40f8-9da0-a07b34cf1d89 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:09.952858 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f947cd3-d400-40f8-9da0-a07b34cf1d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:09.971027 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-8f947cd3-d400-40f8-9da0-a07b34cf1d89 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:09.972842 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-8f947cd3-d400-40f8-9da0-a07b34cf1d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:10.017224 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690bbc0f-1ebf-4ed6-a211-d1c9c2e9521c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:10.019084 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690bbc0f-1ebf-4ed6-a211-d1c9c2e9521c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:10.153145 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 190/378] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:10.159044 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690bbc0f-1ebf-4ed6-a211-d1c9c2e9521c None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:10.164753 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 189/379] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:10.200124 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f947cd3-d400-40f8-9da0-a07b34cf1d89 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:10.211145 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:10.291942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690bbc0f-1ebf-4ed6-a211-d1c9c2e9521c None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:10.319353 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:10.322314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f947cd3-d400-40f8-9da0-a07b34cf1d89 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:10.332178 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6534fd78-7405-4433-9439-9f2994399b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:10.333047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534fd78-7405-4433-9439-9f2994399b96 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:10.333332 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534fd78-7405-4433-9439-9f2994399b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:10.333609 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534fd78-7405-4433-9439-9f2994399b96 None admin] PATH_INFO: `/v3/projects/66e8228804bb44bd89c3894562c27b12` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:10.334835 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6534fd78-7405-4433-9439-9f2994399b96 None admin] RBAC: Authorizing `identity:get_project(project_id=66e8228804bb44bd89c3894562c27b12)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:10.337404 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6534fd78-7405-4433-9439-9f2994399b96 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:10.345183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 191/380] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:10 2022] GET /identity/v3/projects/66e8228804bb44bd89c3894562c27b12 => generated 344 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:10.346060 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6534fd78-7405-4433-9439-9f2994399b96 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:10.367763 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf8764-c44d-4541-b22e-75d47508c55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:10.368578 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf8764-c44d-4541-b22e-75d47508c55c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:10.368863 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf8764-c44d-4541-b22e-75d47508c55c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:10.369139 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf8764-c44d-4541-b22e-75d47508c55c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:10.391171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbf8764-c44d-4541-b22e-75d47508c55c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:10.399152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-7dbf8764-c44d-4541-b22e-75d47508c55c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:10.399152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:10.399152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:10.399152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:10.399152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:10.399152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:10.399152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:10.399152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:10.399152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:10.399152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:10.399152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:10.399152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:10.399152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:10.399152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:10.399152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:10.399152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:10.400011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:10.400011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:10.400011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:10.400011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:10.400011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:10.400011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:10.400011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:10.400011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:10.400011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:10.400011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:10.400011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:10.400011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:10.400011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:10.400011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:10.400011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:10.400011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:10.400961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:10.400961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:10.400961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:10.400961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:10.400961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:10.400961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:10.400961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:10.400961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:10.400961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:10.400961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:10.400961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:10.400961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:10.400961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 190/381] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:10.413470 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf8764-c44d-4541-b22e-75d47508c55c None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:10.528623 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6534fd78-7405-4433-9439-9f2994399b96 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:10.554688 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:10.588497 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf8764-c44d-4541-b22e-75d47508c55c None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:10.598772 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80315736-d270-49e8-9693-aa4d55f437e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:10.599691 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80315736-d270-49e8-9693-aa4d55f437e9 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:10.599976 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80315736-d270-49e8-9693-aa4d55f437e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:10.600276 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80315736-d270-49e8-9693-aa4d55f437e9 None admin] PATH_INFO: `/v3/groups/193167e463a748b8a7bdc11746687f39` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:10.612222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80315736-d270-49e8-9693-aa4d55f437e9 None admin] RBAC: Authorizing `identity:get_group(group_id=193167e463a748b8a7bdc11746687f39)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:10.613682 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80315736-d270-49e8-9693-aa4d55f437e9 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:10.615436 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 192/382] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Jun 15 14:29:10 2022] GET /identity/v3/groups/193167e463a748b8a7bdc11746687f39 => generated 229 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:10.622984 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80315736-d270-49e8-9693-aa4d55f437e9 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:10.644268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff685bb-3568-4db2-8069-3272ff88ef17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:10.645292 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff685bb-3568-4db2-8069-3272ff88ef17 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:10.645807 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff685bb-3568-4db2-8069-3272ff88ef17 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:10.646667 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff685bb-3568-4db2-8069-3272ff88ef17 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:10.647670 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff685bb-3568-4db2-8069-3272ff88ef17 None admin] RBAC: Adding query filter params (name=member) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:10.648205 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff685bb-3568-4db2-8069-3272ff88ef17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:10.650363 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff685bb-3568-4db2-8069-3272ff88ef17 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:10.663426 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 191/383] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Jun 15 14:29:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:10.672027 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff685bb-3568-4db2-8069-3272ff88ef17 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:10.719390 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:10.772001 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80315736-d270-49e8-9693-aa4d55f437e9 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:10.808838 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d9ba2e-ae0e-4e68-b25e-72805d645147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:10.809450 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9ba2e-ae0e-4e68-b25e-72805d645147 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:10.809734 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9ba2e-ae0e-4e68-b25e-72805d645147 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:10.810010 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9ba2e-ae0e-4e68-b25e-72805d645147 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:10.810838 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d9ba2e-ae0e-4e68-b25e-72805d645147 None admin] RBAC: Adding query filter params (group.id=193167e463a748b8a7bdc11746687f39, role.id=ed1cfa2fcf724a4d84f5acea100930c7, scope.project.id=66e8228804bb44bd89c3894562c27b12) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:10.811222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d9ba2e-ae0e-4e68-b25e-72805d645147 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:10.812893 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d9ba2e-ae0e-4e68-b25e-72805d645147 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:10.819106 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 193/384] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Jun 15 14:29:10 2022] GET /identity/v3/role_assignments?group.id=193167e463a748b8a7bdc11746687f39&scope.project.id=66e8228804bb44bd89c3894562c27b12&role.id=ed1cfa2fcf724a4d84f5acea100930c7 => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:10.923289 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff685bb-3568-4db2-8069-3272ff88ef17 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:10.959488 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18341ffb-e812-44c3-9afd-ac060c66c584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:10.960349 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18341ffb-e812-44c3-9afd-ac060c66c584 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:10.960630 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18341ffb-e812-44c3-9afd-ac060c66c584 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:10.960901 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18341ffb-e812-44c3-9afd-ac060c66c584 None admin] PATH_INFO: `/v3/groups/42a71e0d44eb4e3b8d35e0c3b45e9bc2` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:10.962093 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18341ffb-e812-44c3-9afd-ac060c66c584 None admin] RBAC: Authorizing `identity:get_group(group_id=42a71e0d44eb4e3b8d35e0c3b45e9bc2)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:10.963588 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18341ffb-e812-44c3-9afd-ac060c66c584 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:10.965619 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 192/385] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Jun 15 14:29:10 2022] GET /identity/v3/groups/42a71e0d44eb4e3b8d35e0c3b45e9bc2 => generated 226 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:10.971194 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d9ba2e-ae0e-4e68-b25e-72805d645147 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:11.018587 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:11.090188 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d9ba2e-ae0e-4e68-b25e-72805d645147 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:11.125574 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32224372-cdd0-4152-9467-6fda6aa5a6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:11.126441 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32224372-cdd0-4152-9467-6fda6aa5a6d0 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:11.126725 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32224372-cdd0-4152-9467-6fda6aa5a6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:11.127020 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32224372-cdd0-4152-9467-6fda6aa5a6d0 None admin] PATH_INFO: `/v3/projects/8f586cc781db4f2b96f13e85c4851e5a` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:11.128233 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32224372-cdd0-4152-9467-6fda6aa5a6d0 None admin] RBAC: Authorizing `identity:get_project(project_id=8f586cc781db4f2b96f13e85c4851e5a)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:11.129688 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32224372-cdd0-4152-9467-6fda6aa5a6d0 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:11.131734 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 194/386] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:10 2022] GET /identity/v3/projects/8f586cc781db4f2b96f13e85c4851e5a => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:11.137681 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18341ffb-e812-44c3-9afd-ac060c66c584 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:11.271940 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:11.287731 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18341ffb-e812-44c3-9afd-ac060c66c584 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:11.322257 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5b6669-f4b2-4f81-9d53-0a011719301d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:11.323172 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b6669-f4b2-4f81-9d53-0a011719301d None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:11.323475 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b6669-f4b2-4f81-9d53-0a011719301d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:11.323753 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b6669-f4b2-4f81-9d53-0a011719301d None admin] PATH_INFO: `/v3/projects/8f586cc781db4f2b96f13e85c4851e5a/groups/42a71e0d44eb4e3b8d35e0c3b45e9bc2/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:11.325806 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5b6669-f4b2-4f81-9d53-0a011719301d None admin] RBAC: Authorizing `identity:create_grant(project_id=8f586cc781db4f2b96f13e85c4851e5a, group_id=42a71e0d44eb4e3b8d35e0c3b45e9bc2, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:11.327330 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5b6669-f4b2-4f81-9d53-0a011719301d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:11.341790 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 193/387] 10.222.0.45 () {58 vars in 1442 bytes} [Wed Jun 15 14:29:11 2022] PUT /identity/v3/projects/8f586cc781db4f2b96f13e85c4851e5a/groups/42a71e0d44eb4e3b8d35e0c3b45e9bc2/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:29:11.711247 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21373c0-1561-42ef-92e6-19b799c4f168 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:11.711721 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21373c0-1561-42ef-92e6-19b799c4f168 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:11.711721 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21373c0-1561-42ef-92e6-19b799c4f168 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:11.713378 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 195/388] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:11.718669 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464b7a1-3d04-4cb9-83f4-96480c46b5bf None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:11.719228 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464b7a1-3d04-4cb9-83f4-96480c46b5bf None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:11.719700 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464b7a1-3d04-4cb9-83f4-96480c46b5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:11.737728 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-6464b7a1-3d04-4cb9-83f4-96480c46b5bf None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:11.739225 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6464b7a1-3d04-4cb9-83f4-96480c46b5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:11.815094 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 194/389] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:11.831953 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21373c0-1561-42ef-92e6-19b799c4f168 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:11.951670 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21373c0-1561-42ef-92e6-19b799c4f168 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:11.963051 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:11.995988 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64af426-c540-4092-bdca-974a7d63dd4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:11.996798 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64af426-c540-4092-bdca-974a7d63dd4e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:11.997078 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64af426-c540-4092-bdca-974a7d63dd4e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:11.997350 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64af426-c540-4092-bdca-974a7d63dd4e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:12.015915 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64af426-c540-4092-bdca-974a7d63dd4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:12.024764 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-d64af426-c540-4092-bdca-974a7d63dd4e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:12.024764 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:12.024764 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:12.024764 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:12.024764 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:12.024764 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:12.024764 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:12.024764 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:12.024764 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:12.024764 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:12.024764 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:12.024764 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:12.024764 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:12.024764 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:12.024764 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:12.024764 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:12.025637 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:12.025637 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:12.025637 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:12.025637 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:12.025637 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:12.025637 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:12.025637 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:12.025637 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:12.025637 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:12.025637 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:12.025637 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:12.025637 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:12.025637 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:12.025637 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:12.025637 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:12.025637 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:12.026415 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:12.026415 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:12.026415 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:12.026415 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:12.026415 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:12.026415 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:12.026415 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:12.026415 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:12.026415 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:12.026415 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:12.026415 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:12.026415 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:12.026415 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 196/390] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Jun 15 14:29:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:12.032915 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464b7a1-3d04-4cb9-83f4-96480c46b5bf None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:12.129988 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:12.154704 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464b7a1-3d04-4cb9-83f4-96480c46b5bf None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:12.189347 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf1398-5cc0-491f-8d71-6c7c180b3d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:12.190019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf1398-5cc0-491f-8d71-6c7c180b3d05 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:12.190299 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf1398-5cc0-491f-8d71-6c7c180b3d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:12.190572 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf1398-5cc0-491f-8d71-6c7c180b3d05 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:12.191349 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf1398-5cc0-491f-8d71-6c7c180b3d05 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:12.191652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf1398-5cc0-491f-8d71-6c7c180b3d05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:12.193020 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf1398-5cc0-491f-8d71-6c7c180b3d05 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:12.206211 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 195/391] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:12.207563 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64af426-c540-4092-bdca-974a7d63dd4e None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:12.341534 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64af426-c540-4092-bdca-974a7d63dd4e None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:12.389402 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ba3d7-9aaa-46e2-b22a-682bae6c015a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:12.390258 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ba3d7-9aaa-46e2-b22a-682bae6c015a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:12.390537 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ba3d7-9aaa-46e2-b22a-682bae6c015a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:12.390809 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ba3d7-9aaa-46e2-b22a-682bae6c015a None admin] PATH_INFO: `/v3/groups/42a71e0d44eb4e3b8d35e0c3b45e9bc2` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:12.395691 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9ba3d7-9aaa-46e2-b22a-682bae6c015a None admin] RBAC: Authorizing `identity:get_group(group_id=42a71e0d44eb4e3b8d35e0c3b45e9bc2)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:12.397132 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9ba3d7-9aaa-46e2-b22a-682bae6c015a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:12.398926 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 197/392] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Jun 15 14:29:12 2022] GET /identity/v3/groups/42a71e0d44eb4e3b8d35e0c3b45e9bc2 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:12.404322 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf1398-5cc0-491f-8d71-6c7c180b3d05 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:12.559326 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf1398-5cc0-491f-8d71-6c7c180b3d05 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:12.614347 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c7f6b0-cc91-4326-ab37-b388cfff1a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:12.615508 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7f6b0-cc91-4326-ab37-b388cfff1a75 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:12.615949 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7f6b0-cc91-4326-ab37-b388cfff1a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:12.616376 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7f6b0-cc91-4326-ab37-b388cfff1a75 None admin] PATH_INFO: `/v3/projects/8f586cc781db4f2b96f13e85c4851e5a` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:12.623877 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c7f6b0-cc91-4326-ab37-b388cfff1a75 None admin] RBAC: Authorizing `identity:get_project(project_id=8f586cc781db4f2b96f13e85c4851e5a)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:12.625616 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c7f6b0-cc91-4326-ab37-b388cfff1a75 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:12.627590 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 196/393] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:12 2022] GET /identity/v3/projects/8f586cc781db4f2b96f13e85c4851e5a => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:12.633349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ba3d7-9aaa-46e2-b22a-682bae6c015a None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:12.665180 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d645c34-f00d-434e-bc70-03ec25c599f6 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:12.665709 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d645c34-f00d-434e-bc70-03ec25c599f6 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:12.666149 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d645c34-f00d-434e-bc70-03ec25c599f6 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:12.667735 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 197/394] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:29:12.672911 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b7003-3d15-4f32-9902-9aa1d5ed5cdd None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:12.673204 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b7003-3d15-4f32-9902-9aa1d5ed5cdd None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:12.673486 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b7003-3d15-4f32-9902-9aa1d5ed5cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:12.692398 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-856b7003-3d15-4f32-9902-9aa1d5ed5cdd None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:12.694683 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-856b7003-3d15-4f32-9902-9aa1d5ed5cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:12.696840 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:12.700445 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:12.774640 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ba3d7-9aaa-46e2-b22a-682bae6c015a None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:12.786512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 198/395] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:29:12.788445 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9a26f-c071-45a2-b236-50f3f480634d None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:12.788992 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9a26f-c071-45a2-b236-50f3f480634d None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:12.789479 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9a26f-c071-45a2-b236-50f3f480634d None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:12.791218 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 199/396] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:12.796321 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47ae27-107d-4a05-a611-3009638d235f None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:12.797077 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47ae27-107d-4a05-a611-3009638d235f None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:12.797546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47ae27-107d-4a05-a611-3009638d235f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:12.827131 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-2e47ae27-107d-4a05-a611-3009638d235f None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:12.829275 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-2e47ae27-107d-4a05-a611-3009638d235f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:12.832341 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646361c6-66f3-4a1d-b143-7a8cd7690fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:12.833468 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646361c6-66f3-4a1d-b143-7a8cd7690fd0 None admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:12.833801 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646361c6-66f3-4a1d-b143-7a8cd7690fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:12.834118 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646361c6-66f3-4a1d-b143-7a8cd7690fd0 None admin] PATH_INFO: `/v3/projects/8f586cc781db4f2b96f13e85c4851e5a/groups/42a71e0d44eb4e3b8d35e0c3b45e9bc2/roles/6b6353b49c6a41b1a157f8997fc52d9a` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:12.836615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646361c6-66f3-4a1d-b143-7a8cd7690fd0 None admin] RBAC: Authorizing `identity:create_grant(project_id=8f586cc781db4f2b96f13e85c4851e5a, group_id=42a71e0d44eb4e3b8d35e0c3b45e9bc2, role_id=6b6353b49c6a41b1a157f8997fc52d9a)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:12.839014 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646361c6-66f3-4a1d-b143-7a8cd7690fd0 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:12.863023 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 198/397] 10.222.0.45 () {58 vars in 1442 bytes} [Wed Jun 15 14:29:12 2022] PUT /identity/v3/projects/8f586cc781db4f2b96f13e85c4851e5a/groups/42a71e0d44eb4e3b8d35e0c3b45e9bc2/roles/6b6353b49c6a41b1a157f8997fc52d9a => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:29:12.863023 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646361c6-66f3-4a1d-b143-7a8cd7690fd0 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:12.979374 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 200/398] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:13.017533 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47ae27-107d-4a05-a611-3009638d235f None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:13.047382 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:13.051270 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646361c6-66f3-4a1d-b143-7a8cd7690fd0 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:13.097808 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709da5b8-e5fc-498d-91b2-275f30e8582f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:13.098842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709da5b8-e5fc-498d-91b2-275f30e8582f None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:13.099040 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709da5b8-e5fc-498d-91b2-275f30e8582f None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:13.099320 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709da5b8-e5fc-498d-91b2-275f30e8582f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:13.109527 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:13.117912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709da5b8-e5fc-498d-91b2-275f30e8582f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:13.132612 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-709da5b8-e5fc-498d-91b2-275f30e8582f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:13.132612 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:13.132612 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:13.132612 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:13.132612 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:13.132612 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:13.132612 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:13.132612 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:13.132612 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:13.132612 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:13.132612 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:13.132612 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:13.132612 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:13.132612 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:13.132612 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:13.132612 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:13.133549 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:13.133549 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:13.133549 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:13.133549 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:13.133549 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:13.133549 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:13.133549 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:13.133549 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:13.133549 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:13.133549 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:13.133549 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:13.133549 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:13.133549 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:13.133549 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:13.133549 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:13.133549 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:13.134317 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:13.134317 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:13.134317 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:13.134317 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:13.134317 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:13.134317 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:13.134317 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:13.134317 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:13.134317 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:13.134317 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:13.134317 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:13.134317 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:13.137688 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 199/399] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:13.145026 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709da5b8-e5fc-498d-91b2-275f30e8582f None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:13.158329 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47ae27-107d-4a05-a611-3009638d235f None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:13.197465 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7fab7-c1ea-498e-a82b-03cd1aee4374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:13.198427 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7fab7-c1ea-498e-a82b-03cd1aee4374 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:13.199365 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7fab7-c1ea-498e-a82b-03cd1aee4374 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:13.199728 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7fab7-c1ea-498e-a82b-03cd1aee4374 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:13.211062 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac7fab7-c1ea-498e-a82b-03cd1aee4374 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:13.220604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-bac7fab7-c1ea-498e-a82b-03cd1aee4374 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:13.220604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:13.220604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:13.220604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:13.220604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:13.220604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:13.220604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:13.220604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:13.220604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:13.220604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:13.220604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:13.220604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:13.220604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:13.220604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:13.220604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:13.220604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:13.221610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:13.221610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:13.221610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:13.221610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:13.221610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:13.221610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:13.221610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:13.221610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:13.221610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:13.221610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:13.221610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:13.221610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:13.221610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:13.221610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:13.221610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:13.221610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:13.222388 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:13.222388 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:13.222388 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:13.222388 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:13.222388 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:13.222388 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:13.222388 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:13.222388 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:13.222388 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:13.222388 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:13.222388 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:13.222388 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:13.227136 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 201/400] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Jun 15 14:29:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:13.229072 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7fab7-c1ea-498e-a82b-03cd1aee4374 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:13.378705 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7fab7-c1ea-498e-a82b-03cd1aee4374 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:13.381347 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709da5b8-e5fc-498d-91b2-275f30e8582f None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:13.408828 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:13.419477 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b8b4de-83af-4ed2-844f-f576d6bee868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:13.420204 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8b4de-83af-4ed2-844f-f576d6bee868 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:13.420496 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8b4de-83af-4ed2-844f-f576d6bee868 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:13.421297 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8b4de-83af-4ed2-844f-f576d6bee868 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:13.422068 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b8b4de-83af-4ed2-844f-f576d6bee868 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:13.422384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b8b4de-83af-4ed2-844f-f576d6bee868 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:13.424162 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b8b4de-83af-4ed2-844f-f576d6bee868 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:13.437089 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 202/401] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:13.441922 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08633e0b-6b64-4c66-ab81-469de14c9f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:13.442596 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08633e0b-6b64-4c66-ab81-469de14c9f30 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:13.442881 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08633e0b-6b64-4c66-ab81-469de14c9f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:13.443182 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08633e0b-6b64-4c66-ab81-469de14c9f30 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:13.443946 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08633e0b-6b64-4c66-ab81-469de14c9f30 None admin] RBAC: Adding query filter params (name=member) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:13.444247 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08633e0b-6b64-4c66-ab81-469de14c9f30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:13.445646 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08633e0b-6b64-4c66-ab81-469de14c9f30 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:13.448442 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b8b4de-83af-4ed2-844f-f576d6bee868 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:13.464889 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 200/402] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Jun 15 14:29:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:13.470631 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08633e0b-6b64-4c66-ab81-469de14c9f30 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:13.507188 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:13.600820 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b8b4de-83af-4ed2-844f-f576d6bee868 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:13.607165 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08633e0b-6b64-4c66-ab81-469de14c9f30 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:13.639229 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b89c0-d6d1-4077-a6ea-49a0f05ac42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:13.640073 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b89c0-d6d1-4077-a6ea-49a0f05ac42c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:13.640355 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b89c0-d6d1-4077-a6ea-49a0f05ac42c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:13.640630 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b89c0-d6d1-4077-a6ea-49a0f05ac42c None admin] PATH_INFO: `/v3/groups/42a71e0d44eb4e3b8d35e0c3b45e9bc2` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:13.642785 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b89c0-d6d1-4077-a6ea-49a0f05ac42c None admin] RBAC: Authorizing `identity:get_group(group_id=42a71e0d44eb4e3b8d35e0c3b45e9bc2)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:13.644554 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b89c0-d6d1-4077-a6ea-49a0f05ac42c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:13.646495 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 203/403] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Jun 15 14:29:13 2022] GET /identity/v3/groups/42a71e0d44eb4e3b8d35e0c3b45e9bc2 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:13.646800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8a53d0-e464-46bb-8f53-8c5545f10352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:13.647869 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a53d0-e464-46bb-8f53-8c5545f10352 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:13.647988 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cad7d-3e00-4831-8013-57cb68e18696 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:13.648150 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cad7d-3e00-4831-8013-57cb68e18696 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:13.648429 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cad7d-3e00-4831-8013-57cb68e18696 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:13.648778 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a53d0-e464-46bb-8f53-8c5545f10352 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:13.649222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a53d0-e464-46bb-8f53-8c5545f10352 None admin] PATH_INFO: `/v3/groups/42a71e0d44eb4e3b8d35e0c3b45e9bc2` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:13.653769 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 204/404] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:13.653769 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2cad7d-3e00-4831-8013-57cb68e18696 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:13.654796 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8a53d0-e464-46bb-8f53-8c5545f10352 None admin] RBAC: Authorizing `identity:get_group(group_id=42a71e0d44eb4e3b8d35e0c3b45e9bc2)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:13.656461 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8a53d0-e464-46bb-8f53-8c5545f10352 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:13.658577 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 201/405] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Jun 15 14:29:13 2022] GET /identity/v3/groups/42a71e0d44eb4e3b8d35e0c3b45e9bc2 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:13.664122 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2ab9b-6309-4d42-8507-0477bee8217b None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:13.664679 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2ab9b-6309-4d42-8507-0477bee8217b None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:13.665158 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2ab9b-6309-4d42-8507-0477bee8217b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:13.687017 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-e1d2ab9b-6309-4d42-8507-0477bee8217b None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:13.688787 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e1d2ab9b-6309-4d42-8507-0477bee8217b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:13.727359 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:13.769383 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2cad7d-3e00-4831-8013-57cb68e18696 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:13.779835 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 202/406] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:13.779835 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2ab9b-6309-4d42-8507-0477bee8217b None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:13.799890 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:13.834472 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756fd18b-2caf-4f3e-b09c-04b0a3f96250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:13.835741 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756fd18b-2caf-4f3e-b09c-04b0a3f96250 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:13.836346 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756fd18b-2caf-4f3e-b09c-04b0a3f96250 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:13.836821 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756fd18b-2caf-4f3e-b09c-04b0a3f96250 None admin] PATH_INFO: `/v3/projects/59af46f68a07464d897b0c71ece3f2f4` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:13.838520 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756fd18b-2caf-4f3e-b09c-04b0a3f96250 None admin] RBAC: Authorizing `identity:get_project(project_id=59af46f68a07464d897b0c71ece3f2f4)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:13.840562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756fd18b-2caf-4f3e-b09c-04b0a3f96250 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:13.842941 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 205/407] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:13 2022] GET /identity/v3/projects/59af46f68a07464d897b0c71ece3f2f4 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:13.844094 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756fd18b-2caf-4f3e-b09c-04b0a3f96250 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:13.878550 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2ab9b-6309-4d42-8507-0477bee8217b None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:13.913931 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12169a5-c670-4c25-b261-5aeda777e135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:13.914709 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12169a5-c670-4c25-b261-5aeda777e135 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:13.915003 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12169a5-c670-4c25-b261-5aeda777e135 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:13.915286 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12169a5-c670-4c25-b261-5aeda777e135 None admin] PATH_INFO: `/v3/projects/59af46f68a07464d897b0c71ece3f2f4` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:13.916456 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12169a5-c670-4c25-b261-5aeda777e135 None admin] RBAC: Authorizing `identity:get_project(project_id=59af46f68a07464d897b0c71ece3f2f4)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:13.917934 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12169a5-c670-4c25-b261-5aeda777e135 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:13.924456 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 203/408] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:13 2022] GET /identity/v3/projects/59af46f68a07464d897b0c71ece3f2f4 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:13.924456 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12169a5-c670-4c25-b261-5aeda777e135 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:13.973903 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756fd18b-2caf-4f3e-b09c-04b0a3f96250 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:14.004252 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:14.016136 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3133fa09-b8b0-4d01-9f87-dd023e4152db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:14.017316 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133fa09-b8b0-4d01-9f87-dd023e4152db None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:14.017812 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133fa09-b8b0-4d01-9f87-dd023e4152db None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:14.018684 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133fa09-b8b0-4d01-9f87-dd023e4152db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:14.025514 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:14.030363 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3133fa09-b8b0-4d01-9f87-dd023e4152db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:14.038133 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-3133fa09-b8b0-4d01-9f87-dd023e4152db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:29:14.038133 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:14.038133 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:14.038133 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:14.038133 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:14.038133 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:14.038133 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:14.038133 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:14.038133 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:14.038133 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:14.038133 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:14.038133 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:14.038133 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:14.038133 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:14.038133 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:14.038133 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:14.039014 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:14.039014 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:14.039014 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:14.039014 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:14.039014 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:14.039014 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:14.039014 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:14.039014 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:14.039014 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:14.039014 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:14.039014 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:14.039014 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:14.039014 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:14.039014 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:14.039014 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:14.039014 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:14.039793 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:14.039793 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:14.039793 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:14.039793 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:14.039793 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:14.039793 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:14.039793 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:14.039793 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:14.039793 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:14.039793 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:14.039793 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:29:14.039793 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:14.043686 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12169a5-c670-4c25-b261-5aeda777e135 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:14.044514 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 206/409] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:29:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:14.047653 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3133fa09-b8b0-4d01-9f87-dd023e4152db None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:14.083093 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72706cfa-54ee-4455-bfb8-c89623d64b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:14.084334 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72706cfa-54ee-4455-bfb8-c89623d64b86 None admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:14.084896 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72706cfa-54ee-4455-bfb8-c89623d64b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:14.085343 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72706cfa-54ee-4455-bfb8-c89623d64b86 None admin] PATH_INFO: `/v3/projects/59af46f68a07464d897b0c71ece3f2f4/groups/42a71e0d44eb4e3b8d35e0c3b45e9bc2/roles/6b6353b49c6a41b1a157f8997fc52d9a` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:14.087724 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72706cfa-54ee-4455-bfb8-c89623d64b86 None admin] RBAC: Authorizing `identity:create_grant(project_id=59af46f68a07464d897b0c71ece3f2f4, group_id=42a71e0d44eb4e3b8d35e0c3b45e9bc2, role_id=6b6353b49c6a41b1a157f8997fc52d9a)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:14.089416 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72706cfa-54ee-4455-bfb8-c89623d64b86 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:14.104993 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 204/410] 10.222.0.45 () {58 vars in 1442 bytes} [Wed Jun 15 14:29:13 2022] PUT /identity/v3/projects/59af46f68a07464d897b0c71ece3f2f4/groups/42a71e0d44eb4e3b8d35e0c3b45e9bc2/roles/6b6353b49c6a41b1a157f8997fc52d9a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:29:14.106682 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2a5be-8898-4b76-aab2-a1d9b90566e8 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:14.107212 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2a5be-8898-4b76-aab2-a1d9b90566e8 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:14.107700 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2a5be-8898-4b76-aab2-a1d9b90566e8 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:14.109439 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 205/411] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:14.110431 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e2a5be-8898-4b76-aab2-a1d9b90566e8 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:14.189892 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3133fa09-b8b0-4d01-9f87-dd023e4152db None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:14.244032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79356fe8-9301-4ef0-8269-ec83e87ac5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:14.245278 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79356fe8-9301-4ef0-8269-ec83e87ac5c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:14.246186 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79356fe8-9301-4ef0-8269-ec83e87ac5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:14.246472 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79356fe8-9301-4ef0-8269-ec83e87ac5c0 None admin] PATH_INFO: `/v3/projects/59af46f68a07464d897b0c71ece3f2f4/groups/42a71e0d44eb4e3b8d35e0c3b45e9bc2/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:14.249109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79356fe8-9301-4ef0-8269-ec83e87ac5c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=59af46f68a07464d897b0c71ece3f2f4, group_id=42a71e0d44eb4e3b8d35e0c3b45e9bc2, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:14.251368 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79356fe8-9301-4ef0-8269-ec83e87ac5c0 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:14.266869 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e2a5be-8898-4b76-aab2-a1d9b90566e8 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:14.271549 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 207/412] 10.222.0.45 () {58 vars in 1442 bytes} [Wed Jun 15 14:29:14 2022] PUT /identity/v3/projects/59af46f68a07464d897b0c71ece3f2f4/groups/42a71e0d44eb4e3b8d35e0c3b45e9bc2/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:29:14.273008 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735a84e-c860-4bbc-ad55-066ebd7b9144 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:14.273418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735a84e-c860-4bbc-ad55-066ebd7b9144 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:14.273746 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735a84e-c860-4bbc-ad55-066ebd7b9144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:14.302820 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-7735a84e-c860-4bbc-ad55-066ebd7b9144 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:14.304198 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-7735a84e-c860-4bbc-ad55-066ebd7b9144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:14.316462 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c84ef-d9e0-4a59-b206-77cab06b2be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:14.317346 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c84ef-d9e0-4a59-b206-77cab06b2be2 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:14.317731 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c84ef-d9e0-4a59-b206-77cab06b2be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:14.318134 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c84ef-d9e0-4a59-b206-77cab06b2be2 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:14.319079 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7c84ef-d9e0-4a59-b206-77cab06b2be2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:14.319491 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7c84ef-d9e0-4a59-b206-77cab06b2be2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:14.321016 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7c84ef-d9e0-4a59-b206-77cab06b2be2 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:14.332048 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 206/413] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:14.337037 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c84ef-d9e0-4a59-b206-77cab06b2be2 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:14.355262 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:14.398640 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:14.443467 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 208/414] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:14.475664 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735a84e-c860-4bbc-ad55-066ebd7b9144 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:14.508430 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c84ef-d9e0-4a59-b206-77cab06b2be2 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:14.554470 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92544224-687d-43cb-a733-4b65995387b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:14.555148 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92544224-687d-43cb-a733-4b65995387b8 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:14.555871 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92544224-687d-43cb-a733-4b65995387b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:14.556316 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92544224-687d-43cb-a733-4b65995387b8 None admin] PATH_INFO: `/v3/groups/193167e463a748b8a7bdc11746687f39` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:14.558040 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92544224-687d-43cb-a733-4b65995387b8 None admin] RBAC: Authorizing `identity:get_group(group_id=193167e463a748b8a7bdc11746687f39)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:14.559860 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92544224-687d-43cb-a733-4b65995387b8 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:14.561619 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 207/415] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Jun 15 14:29:14 2022] GET /identity/v3/groups/193167e463a748b8a7bdc11746687f39 => generated 229 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:14.567568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92544224-687d-43cb-a733-4b65995387b8 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:14.624302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735a84e-c860-4bbc-ad55-066ebd7b9144 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:14.676405 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a7227d-7787-4d8f-9123-a758265bddec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:14.677736 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a7227d-7787-4d8f-9123-a758265bddec None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:14.678228 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a7227d-7787-4d8f-9123-a758265bddec None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:14.678737 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a7227d-7787-4d8f-9123-a758265bddec None admin] PATH_INFO: `/v3/roles/member` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:14.701392 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:14.704295 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a7227d-7787-4d8f-9123-a758265bddec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:14.718213 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-74a7227d-7787-4d8f-9123-a758265bddec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:14.718213 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:14.718213 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:14.718213 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:14.718213 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:14.718213 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:14.718213 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:14.718213 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:14.718213 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:14.718213 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:14.718213 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:14.718213 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:14.718213 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:14.718213 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:14.718213 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:14.718213 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:14.719164 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:14.719164 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:14.719164 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:14.719164 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:14.719164 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:14.719164 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:14.719164 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:14.719164 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:14.719164 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:14.719164 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:14.719164 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:14.719164 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:14.719164 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:14.719164 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:14.719164 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:14.719164 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:14.719938 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:14.719938 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:14.719938 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:14.719938 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:14.719938 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:14.719938 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:14.719938 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:14.719938 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:14.719938 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:14.719938 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:14.719938 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:14.719938 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:14.725954 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 209/416] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:29:14.732108 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a7227d-7787-4d8f-9123-a758265bddec None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:14.750639 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92544224-687d-43cb-a733-4b65995387b8 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:14.786527 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:14.824732 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6038f5ff-5d9c-4238-bcaa-69ce71583349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:14.826280 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038f5ff-5d9c-4238-bcaa-69ce71583349 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:14.826763 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038f5ff-5d9c-4238-bcaa-69ce71583349 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:14.827265 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038f5ff-5d9c-4238-bcaa-69ce71583349 None admin] PATH_INFO: `/v3/projects/66e8228804bb44bd89c3894562c27b12` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:14.835014 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6038f5ff-5d9c-4238-bcaa-69ce71583349 None admin] RBAC: Authorizing `identity:get_project(project_id=66e8228804bb44bd89c3894562c27b12)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:14.836762 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6038f5ff-5d9c-4238-bcaa-69ce71583349 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:14.839031 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 208/417] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:14 2022] GET /identity/v3/projects/66e8228804bb44bd89c3894562c27b12 => generated 344 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:14.845402 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6038f5ff-5d9c-4238-bcaa-69ce71583349 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:14.858612 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a7227d-7787-4d8f-9123-a758265bddec None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:14.894708 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb7d93a-fcae-4040-bcb1-4498b66bffeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:14.895474 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7d93a-fcae-4040-bcb1-4498b66bffeb None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:14.895761 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7d93a-fcae-4040-bcb1-4498b66bffeb None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:14.896038 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7d93a-fcae-4040-bcb1-4498b66bffeb None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:14.899025 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb7d93a-fcae-4040-bcb1-4498b66bffeb None admin] RBAC: Adding query filter params (name=member) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:14.899025 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb7d93a-fcae-4040-bcb1-4498b66bffeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:14.899025 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb7d93a-fcae-4040-bcb1-4498b66bffeb None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:14.908254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 210/418] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Jun 15 14:29:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:14.913132 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb7d93a-fcae-4040-bcb1-4498b66bffeb None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:15.057417 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6038f5ff-5d9c-4238-bcaa-69ce71583349 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:15.062826 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:15.071733 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb7d93a-fcae-4040-bcb1-4498b66bffeb None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:15.110375 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8841b0-77ae-4a20-b71d-284c7157570c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:15.111604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8841b0-77ae-4a20-b71d-284c7157570c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:15.111903 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8841b0-77ae-4a20-b71d-284c7157570c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:15.112238 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8841b0-77ae-4a20-b71d-284c7157570c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:15.164244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ca008d-7b53-4ec8-9a93-db7986f28f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:15.165138 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca008d-7b53-4ec8-9a93-db7986f28f3c None admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:15.165421 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca008d-7b53-4ec8-9a93-db7986f28f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:15.165699 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca008d-7b53-4ec8-9a93-db7986f28f3c None admin] PATH_INFO: `/v3/projects/66e8228804bb44bd89c3894562c27b12/groups/193167e463a748b8a7bdc11746687f39/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:15.172339 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ca008d-7b53-4ec8-9a93-db7986f28f3c None admin] RBAC: Authorizing `identity:create_grant(project_id=66e8228804bb44bd89c3894562c27b12, group_id=193167e463a748b8a7bdc11746687f39, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:15.173775 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ca008d-7b53-4ec8-9a93-db7986f28f3c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:15.191528 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8841b0-77ae-4a20-b71d-284c7157570c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:15.193698 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8841b0-77ae-4a20-b71d-284c7157570c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:15.219933 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 209/419] 10.222.0.45 () {58 vars in 1442 bytes} [Wed Jun 15 14:29:14 2022] PUT /identity/v3/projects/66e8228804bb44bd89c3894562c27b12/groups/193167e463a748b8a7bdc11746687f39/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:29:15.301906 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:15.309453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 211/420] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 14:29:15.315450 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ca008d-7b53-4ec8-9a93-db7986f28f3c None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:15.320768 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:15.425063 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645333c0-890a-406b-a6fd-fc6a4f822ef4 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:15.425387 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645333c0-890a-406b-a6fd-fc6a4f822ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:15.425638 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645333c0-890a-406b-a6fd-fc6a4f822ef4 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:15.427737 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 212/421] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:29:15.435617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b528a-c838-43c5-a3c0-d85130f25489 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:15.435948 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b528a-c838-43c5-a3c0-d85130f25489 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:15.436166 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b528a-c838-43c5-a3c0-d85130f25489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:15.456404 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ca008d-7b53-4ec8-9a93-db7986f28f3c None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:15.480742 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-ca0b528a-c838-43c5-a3c0-d85130f25489 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:15.481962 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ca0b528a-c838-43c5-a3c0-d85130f25489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:15.498210 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82520b6-dc8c-4026-b262-a01449a697af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:15.499494 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82520b6-dc8c-4026-b262-a01449a697af None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:15.500003 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82520b6-dc8c-4026-b262-a01449a697af None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:15.500473 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82520b6-dc8c-4026-b262-a01449a697af None admin] PATH_INFO: `/v3/projects/8f586cc781db4f2b96f13e85c4851e5a` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:15.502727 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82520b6-dc8c-4026-b262-a01449a697af None admin] RBAC: Authorizing `identity:get_project(project_id=8f586cc781db4f2b96f13e85c4851e5a)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:15.504892 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82520b6-dc8c-4026-b262-a01449a697af None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:15.507327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 210/422] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:15 2022] GET /identity/v3/projects/8f586cc781db4f2b96f13e85c4851e5a => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:15.513790 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82520b6-dc8c-4026-b262-a01449a697af None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:15.601008 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:15.648861 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 213/423] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:15.663429 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82520b6-dc8c-4026-b262-a01449a697af None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:15.675834 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0b528a-c838-43c5-a3c0-d85130f25489 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:15.722184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a459719-bc27-4d3a-8e08-8a5b46ff0443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:15.723110 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a459719-bc27-4d3a-8e08-8a5b46ff0443 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:15.723575 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a459719-bc27-4d3a-8e08-8a5b46ff0443 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:15.723857 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a459719-bc27-4d3a-8e08-8a5b46ff0443 None admin] PATH_INFO: `/v3/groups/42a71e0d44eb4e3b8d35e0c3b45e9bc2` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:15.725094 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a459719-bc27-4d3a-8e08-8a5b46ff0443 None admin] RBAC: Authorizing `identity:get_group(group_id=42a71e0d44eb4e3b8d35e0c3b45e9bc2)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:15.726535 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a459719-bc27-4d3a-8e08-8a5b46ff0443 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:15.733511 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 211/424] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Jun 15 14:29:15 2022] GET /identity/v3/groups/42a71e0d44eb4e3b8d35e0c3b45e9bc2 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:15.734852 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a459719-bc27-4d3a-8e08-8a5b46ff0443 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:15.850659 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a459719-bc27-4d3a-8e08-8a5b46ff0443 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:15.888053 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21976754-dff7-49fe-9825-2e2a8d2a52c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:15.888634 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21976754-dff7-49fe-9825-2e2a8d2a52c7 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:15.888918 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21976754-dff7-49fe-9825-2e2a8d2a52c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:15.889192 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21976754-dff7-49fe-9825-2e2a8d2a52c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:15.890023 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21976754-dff7-49fe-9825-2e2a8d2a52c7 None admin] RBAC: Adding query filter params (group.id=42a71e0d44eb4e3b8d35e0c3b45e9bc2, role.id=ce0de9b9c9054826b569db895942ad26, scope.project.id=8f586cc781db4f2b96f13e85c4851e5a) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:15.890333 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21976754-dff7-49fe-9825-2e2a8d2a52c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:15.891799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21976754-dff7-49fe-9825-2e2a8d2a52c7 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:15.903121 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 212/425] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Jun 15 14:29:15 2022] GET /identity/v3/role_assignments?group.id=42a71e0d44eb4e3b8d35e0c3b45e9bc2&scope.project.id=8f586cc781db4f2b96f13e85c4851e5a&role.id=ce0de9b9c9054826b569db895942ad26 => generated 611 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:15.903946 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0b528a-c838-43c5-a3c0-d85130f25489 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:15.958693 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:15.971988 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c196fc-0977-499b-af89-383ebfa83a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:15.972822 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c196fc-0977-499b-af89-383ebfa83a27 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:15.973107 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c196fc-0977-499b-af89-383ebfa83a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:15.973382 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c196fc-0977-499b-af89-383ebfa83a27 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:15.992292 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c196fc-0977-499b-af89-383ebfa83a27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:16.001265 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-83c196fc-0977-499b-af89-383ebfa83a27 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:16.001265 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:16.001265 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:16.001265 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:16.001265 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:16.001265 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:16.001265 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:16.001265 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:16.001265 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:16.001265 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:16.001265 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:16.001265 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:16.001265 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:16.001265 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:16.001265 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:16.001265 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:16.002171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:16.002171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:16.002171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:16.002171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:16.002171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:16.002171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:16.002171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:16.002171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:16.002171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:16.002171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:16.002171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:16.002171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:16.002171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:16.002171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:16.002171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:16.002171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:16.002961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:16.002961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:16.002961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:16.002961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:16.002961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:16.002961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:16.002961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:16.002961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:16.002961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:16.002961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:16.002961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:16.002961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:16.002961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 214/426] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Jun 15 14:29:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:29:16.008580 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21976754-dff7-49fe-9825-2e2a8d2a52c7 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:16.012798 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:16.161544 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21976754-dff7-49fe-9825-2e2a8d2a52c7 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:16.196901 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a935ea-0efd-4c02-823e-078c4bc699c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:16.197568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a935ea-0efd-4c02-823e-078c4bc699c5 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:16.197851 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a935ea-0efd-4c02-823e-078c4bc699c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:16.198129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a935ea-0efd-4c02-823e-078c4bc699c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:16.198911 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a935ea-0efd-4c02-823e-078c4bc699c5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:16.199255 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a935ea-0efd-4c02-823e-078c4bc699c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:16.200638 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a935ea-0efd-4c02-823e-078c4bc699c5 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:16.209456 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 213/427] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:16.215466 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c196fc-0977-499b-af89-383ebfa83a27 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:16.320305 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c196fc-0977-499b-af89-383ebfa83a27 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:16.358350 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dca196-2820-44e7-9233-8d9f54f025fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:16.359026 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dca196-2820-44e7-9233-8d9f54f025fb None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:16.359316 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dca196-2820-44e7-9233-8d9f54f025fb None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:16.359598 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dca196-2820-44e7-9233-8d9f54f025fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:16.434267 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:16.448950 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dca196-2820-44e7-9233-8d9f54f025fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:16.452691 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dca196-2820-44e7-9233-8d9f54f025fb None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:16.572629 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 215/428] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:16.580188 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a935ea-0efd-4c02-823e-078c4bc699c5 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:16.650069 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31409020-af1d-432b-85f4-5e489f1a851c None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:16.650707 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31409020-af1d-432b-85f4-5e489f1a851c None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:16.651202 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31409020-af1d-432b-85f4-5e489f1a851c None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:16.653338 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 216/429] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:16.659760 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3b03a-0c4a-4b47-b8fe-30d597582c7b None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:16.660051 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3b03a-0c4a-4b47-b8fe-30d597582c7b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:16.660329 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3b03a-0c4a-4b47-b8fe-30d597582c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:16.660927 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:16.684266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-12a3b03a-0c4a-4b47-b8fe-30d597582c7b None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:16.685807 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-12a3b03a-0c4a-4b47-b8fe-30d597582c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:16.729397 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a935ea-0efd-4c02-823e-078c4bc699c5 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:16.779322 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190fa9d-6e2d-4403-80db-80db0623b506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:16.780509 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190fa9d-6e2d-4403-80db-80db0623b506 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:16.780938 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190fa9d-6e2d-4403-80db-80db0623b506 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:16.781361 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190fa9d-6e2d-4403-80db-80db0623b506 None admin] PATH_INFO: `/v3/projects/8f586cc781db4f2b96f13e85c4851e5a` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:16.782826 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b190fa9d-6e2d-4403-80db-80db0623b506 None admin] RBAC: Authorizing `identity:get_project(project_id=8f586cc781db4f2b96f13e85c4851e5a)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:16.784936 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b190fa9d-6e2d-4403-80db-80db0623b506 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:16.787519 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 214/430] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:16 2022] GET /identity/v3/projects/8f586cc781db4f2b96f13e85c4851e5a => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:16.793132 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190fa9d-6e2d-4403-80db-80db0623b506 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:16.797044 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 217/431] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:29:16.821326 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a3b03a-0c4a-4b47-b8fe-30d597582c7b None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:16.840641 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:16.920799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190fa9d-6e2d-4403-80db-80db0623b506 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:16.939722 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a3b03a-0c4a-4b47-b8fe-30d597582c7b None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:16.962188 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47548583-68e0-48f8-a76a-42e9c7ae298a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:16.963091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47548583-68e0-48f8-a76a-42e9c7ae298a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:16.963380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47548583-68e0-48f8-a76a-42e9c7ae298a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:16.963650 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47548583-68e0-48f8-a76a-42e9c7ae298a None admin] PATH_INFO: `/v3/groups/42a71e0d44eb4e3b8d35e0c3b45e9bc2` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:16.964898 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47548583-68e0-48f8-a76a-42e9c7ae298a None admin] RBAC: Authorizing `identity:get_group(group_id=42a71e0d44eb4e3b8d35e0c3b45e9bc2)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:16.966331 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47548583-68e0-48f8-a76a-42e9c7ae298a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:16.972571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 215/432] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Jun 15 14:29:16 2022] GET /identity/v3/groups/42a71e0d44eb4e3b8d35e0c3b45e9bc2 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:16.972571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23192b2d-825f-4671-999d-b5dcdd180821 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:16.972571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23192b2d-825f-4671-999d-b5dcdd180821 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:16.972571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23192b2d-825f-4671-999d-b5dcdd180821 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:16.972571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 216/433] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:16.975570 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23192b2d-825f-4671-999d-b5dcdd180821 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:16.978242 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:16.983111 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a4080-f79c-49de-8042-fde5dbc7e946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:16.983931 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a4080-f79c-49de-8042-fde5dbc7e946 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:16.984216 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a4080-f79c-49de-8042-fde5dbc7e946 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:16.984495 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a4080-f79c-49de-8042-fde5dbc7e946 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:16.996007 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a4080-f79c-49de-8042-fde5dbc7e946 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:17.004159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-f65a4080-f79c-49de-8042-fde5dbc7e946 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:17.004159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:17.004159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:17.004159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:17.004159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:17.004159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:17.004159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:17.004159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:17.004159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:17.004159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:17.004159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:17.004159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:17.004159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:17.004159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:17.004159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:17.004159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:17.005000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:17.005000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:17.005000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:17.005000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:17.005000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:17.005000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:17.005000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:17.005000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:17.005000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:17.005000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:17.005000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:17.005000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:17.005000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:17.005000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:17.005000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:17.005000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:17.005799 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:17.005799 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:17.005799 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:17.005799 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:17.005799 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:17.005799 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:17.005799 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:17.005799 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:17.005799 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:17.005799 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:17.005799 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:17.005799 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:17.005799 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 218/434] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Jun 15 14:29:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:17.006548 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca61ed-5cee-4398-baa3-f555ac71f923 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:17.007259 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca61ed-5cee-4398-baa3-f555ac71f923 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:17.007259 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca61ed-5cee-4398-baa3-f555ac71f923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:17.029821 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-0cca61ed-5cee-4398-baa3-f555ac71f923 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:17.031214 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-0cca61ed-5cee-4398-baa3-f555ac71f923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:17.098806 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23192b2d-825f-4671-999d-b5dcdd180821 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:17.111198 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 219/435] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:17.111198 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca61ed-5cee-4398-baa3-f555ac71f923 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:17.111829 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:17.141760 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77325a-630c-4ebb-a695-7bff7eaeae6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:17.142461 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77325a-630c-4ebb-a695-7bff7eaeae6b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:17.142752 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77325a-630c-4ebb-a695-7bff7eaeae6b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:17.143056 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77325a-630c-4ebb-a695-7bff7eaeae6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:17.144195 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e77325a-630c-4ebb-a695-7bff7eaeae6b None admin] RBAC: Adding query filter params (group.id=42a71e0d44eb4e3b8d35e0c3b45e9bc2, role.id=6b6353b49c6a41b1a157f8997fc52d9a, scope.project.id=8f586cc781db4f2b96f13e85c4851e5a) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:17.144508 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e77325a-630c-4ebb-a695-7bff7eaeae6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:17.146350 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e77325a-630c-4ebb-a695-7bff7eaeae6b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:17.163125 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 217/436] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Jun 15 14:29:16 2022] GET /identity/v3/role_assignments?group.id=42a71e0d44eb4e3b8d35e0c3b45e9bc2&scope.project.id=8f586cc781db4f2b96f13e85c4851e5a&role.id=6b6353b49c6a41b1a157f8997fc52d9a => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:17.163125 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77325a-630c-4ebb-a695-7bff7eaeae6b None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:17.213168 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca61ed-5cee-4398-baa3-f555ac71f923 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:17.254854 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:17.262783 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9add4-bd8a-46b3-bc6d-c9d2cd019571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:17.263515 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9add4-bd8a-46b3-bc6d-c9d2cd019571 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:17.263801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9add4-bd8a-46b3-bc6d-c9d2cd019571 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:17.264077 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9add4-bd8a-46b3-bc6d-c9d2cd019571 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:17.264844 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9add4-bd8a-46b3-bc6d-c9d2cd019571 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:17.265156 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9add4-bd8a-46b3-bc6d-c9d2cd019571 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:17.266570 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9add4-bd8a-46b3-bc6d-c9d2cd019571 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:17.283167 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 220/437] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:17.285534 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9add4-bd8a-46b3-bc6d-c9d2cd019571 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:17.303655 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77325a-630c-4ebb-a695-7bff7eaeae6b None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:17.343982 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f60a11-7463-4dca-be14-f1cf73a9936a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:17.344777 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f60a11-7463-4dca-be14-f1cf73a9936a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:17.345056 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f60a11-7463-4dca-be14-f1cf73a9936a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:17.345324 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f60a11-7463-4dca-be14-f1cf73a9936a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:17.359017 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f60a11-7463-4dca-be14-f1cf73a9936a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:17.366927 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-99f60a11-7463-4dca-be14-f1cf73a9936a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:17.366927 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:17.366927 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:17.366927 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:17.366927 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:17.366927 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:17.366927 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:17.366927 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:17.366927 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:17.366927 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:17.366927 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:17.366927 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:17.366927 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:17.366927 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:17.366927 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:17.366927 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:17.367958 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:17.367958 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:17.367958 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:17.367958 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:17.367958 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:17.367958 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:17.367958 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:17.367958 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:17.367958 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:17.367958 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:17.367958 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:17.367958 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:17.367958 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:17.367958 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:17.367958 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:17.367958 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:17.368734 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:17.368734 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:17.368734 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:17.368734 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:17.368734 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:17.368734 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:17.368734 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:17.368734 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:17.368734 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:17.368734 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:17.368734 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:17.368734 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:17.371049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 218/438] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:17.374059 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f60a11-7463-4dca-be14-f1cf73a9936a None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:17.384817 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:17.425054 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9add4-bd8a-46b3-bc6d-c9d2cd019571 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:17.474692 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66757e6-93e6-48b5-b311-aac2900562c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:17.474692 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66757e6-93e6-48b5-b311-aac2900562c5 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:17.475104 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66757e6-93e6-48b5-b311-aac2900562c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:17.475286 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66757e6-93e6-48b5-b311-aac2900562c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:17.518323 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f60a11-7463-4dca-be14-f1cf73a9936a None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:17.548893 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:17.567884 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c109f7-77ca-48c0-8889-cfe924d30554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:17.568227 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c109f7-77ca-48c0-8889-cfe924d30554 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:17.568627 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c109f7-77ca-48c0-8889-cfe924d30554 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:17.568907 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c109f7-77ca-48c0-8889-cfe924d30554 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:17.569664 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c109f7-77ca-48c0-8889-cfe924d30554 None admin] RBAC: Adding query filter params (name=member) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:17.569970 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c109f7-77ca-48c0-8889-cfe924d30554 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:17.570488 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66757e6-93e6-48b5-b311-aac2900562c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:17.571370 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c109f7-77ca-48c0-8889-cfe924d30554 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:17.571990 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66757e6-93e6-48b5-b311-aac2900562c5 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:17.581777 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 219/439] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Jun 15 14:29:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:17.587156 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c109f7-77ca-48c0-8889-cfe924d30554 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:17.647542 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:17.684641 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 221/440] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:17.690021 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66757e6-93e6-48b5-b311-aac2900562c5 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:17.692279 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c109f7-77ca-48c0-8889-cfe924d30554 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:17.732239 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f114bbc9-bebe-4f9a-8028-c93b1f2bcb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:17.732895 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114bbc9-bebe-4f9a-8028-c93b1f2bcb9c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:17.733180 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114bbc9-bebe-4f9a-8028-c93b1f2bcb9c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:17.733457 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114bbc9-bebe-4f9a-8028-c93b1f2bcb9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:17.800469 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66757e6-93e6-48b5-b311-aac2900562c5 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:17.801455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:17.815828 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f114bbc9-bebe-4f9a-8028-c93b1f2bcb9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:17.817547 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f114bbc9-bebe-4f9a-8028-c93b1f2bcb9c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:17.835307 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c35b26-9301-4df9-a136-7b2ca28551da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:17.836159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c35b26-9301-4df9-a136-7b2ca28551da None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:17.836444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c35b26-9301-4df9-a136-7b2ca28551da None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:17.836721 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c35b26-9301-4df9-a136-7b2ca28551da None admin] PATH_INFO: `/v3/projects/59af46f68a07464d897b0c71ece3f2f4` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:17.838166 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c35b26-9301-4df9-a136-7b2ca28551da None admin] RBAC: Authorizing `identity:get_project(project_id=59af46f68a07464d897b0c71ece3f2f4)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:17.840252 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c35b26-9301-4df9-a136-7b2ca28551da None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:17.842374 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 222/441] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:17 2022] GET /identity/v3/projects/59af46f68a07464d897b0c71ece3f2f4 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:17.843573 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b9208-b9e9-42c6-8496-c472d8a1053b None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:17.843862 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b9208-b9e9-42c6-8496-c472d8a1053b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:17.844137 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b9208-b9e9-42c6-8496-c472d8a1053b None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:17.845354 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 223/442] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:17.847714 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b9208-b9e9-42c6-8496-c472d8a1053b None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:17.888187 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:17.924335 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 220/443] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 14:29:17.926678 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f45749-3eff-4e4f-b2ab-e1fed51a8715 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:17.926967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f45749-3eff-4e4f-b2ab-e1fed51a8715 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:17.927559 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f45749-3eff-4e4f-b2ab-e1fed51a8715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:17.943151 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-77f45749-3eff-4e4f-b2ab-e1fed51a8715 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:17.944389 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-77f45749-3eff-4e4f-b2ab-e1fed51a8715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:17.949964 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b9208-b9e9-42c6-8496-c472d8a1053b None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:17.983216 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65227867-2f47-475f-802a-16176a14f1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:17.983996 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65227867-2f47-475f-802a-16176a14f1b0 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:17.984276 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65227867-2f47-475f-802a-16176a14f1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:17.984550 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65227867-2f47-475f-802a-16176a14f1b0 None admin] PATH_INFO: `/v3/groups/42a71e0d44eb4e3b8d35e0c3b45e9bc2` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:17.985723 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65227867-2f47-475f-802a-16176a14f1b0 None admin] RBAC: Authorizing `identity:get_group(group_id=42a71e0d44eb4e3b8d35e0c3b45e9bc2)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:17.987201 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65227867-2f47-475f-802a-16176a14f1b0 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:17.988976 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 224/444] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Jun 15 14:29:17 2022] GET /identity/v3/groups/42a71e0d44eb4e3b8d35e0c3b45e9bc2 => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:17.989568 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65227867-2f47-475f-802a-16176a14f1b0 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:18.008713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 221/445] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:18.009560 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f45749-3eff-4e4f-b2ab-e1fed51a8715 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:18.031951 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:18.092540 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65227867-2f47-475f-802a-16176a14f1b0 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:18.100325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:18.110478 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f45749-3eff-4e4f-b2ab-e1fed51a8715 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:18.134296 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7addab9e-17bd-4f77-890a-6fe26e981db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:18.135278 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7addab9e-17bd-4f77-890a-6fe26e981db3 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:18.135631 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7addab9e-17bd-4f77-890a-6fe26e981db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:18.135903 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7addab9e-17bd-4f77-890a-6fe26e981db3 None admin] PATH_INFO: `/v3/projects/59af46f68a07464d897b0c71ece3f2f4` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:18.137124 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7addab9e-17bd-4f77-890a-6fe26e981db3 None admin] RBAC: Authorizing `identity:get_project(project_id=59af46f68a07464d897b0c71ece3f2f4)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:18.138572 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7addab9e-17bd-4f77-890a-6fe26e981db3 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:18.140310 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 225/446] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:17 2022] GET /identity/v3/projects/59af46f68a07464d897b0c71ece3f2f4 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:18.141068 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7addab9e-17bd-4f77-890a-6fe26e981db3 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:18.152739 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ad479-5bc9-43f7-9da8-f70082416b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:18.153393 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ad479-5bc9-43f7-9da8-f70082416b1f None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:18.153682 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ad479-5bc9-43f7-9da8-f70082416b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:18.153962 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ad479-5bc9-43f7-9da8-f70082416b1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:18.154794 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481ad479-5bc9-43f7-9da8-f70082416b1f None admin] RBAC: Adding query filter params (group.id=42a71e0d44eb4e3b8d35e0c3b45e9bc2, role.id=6b6353b49c6a41b1a157f8997fc52d9a, scope.project.id=59af46f68a07464d897b0c71ece3f2f4) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:18.155144 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481ad479-5bc9-43f7-9da8-f70082416b1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:18.156578 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481ad479-5bc9-43f7-9da8-f70082416b1f None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:18.162274 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 222/447] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Jun 15 14:29:18 2022] GET /identity/v3/role_assignments?group.id=42a71e0d44eb4e3b8d35e0c3b45e9bc2&scope.project.id=59af46f68a07464d897b0c71ece3f2f4&role.id=6b6353b49c6a41b1a157f8997fc52d9a => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:18.163188 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ad479-5bc9-43f7-9da8-f70082416b1f None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:18.247398 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7addab9e-17bd-4f77-890a-6fe26e981db3 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:18.259504 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:18.281176 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ad479-5bc9-43f7-9da8-f70082416b1f None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:18.289194 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdaa5b7-f6ba-4660-985b-96e1d15ef3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:18.290007 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdaa5b7-f6ba-4660-985b-96e1d15ef3dd None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:18.290288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdaa5b7-f6ba-4660-985b-96e1d15ef3dd None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:18.290556 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdaa5b7-f6ba-4660-985b-96e1d15ef3dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:18.302762 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdaa5b7-f6ba-4660-985b-96e1d15ef3dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:18.310617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-5bdaa5b7-f6ba-4660-985b-96e1d15ef3dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:29:18.310617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:18.310617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:18.310617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:18.310617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:18.310617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:18.310617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:18.310617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:18.310617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:18.310617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:18.310617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:18.310617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:18.310617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:18.310617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:18.310617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:18.310617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:18.311814 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:18.311814 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:18.311814 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:18.311814 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:18.311814 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:18.311814 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:18.311814 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:18.311814 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:18.311814 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:18.311814 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:18.311814 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:18.311814 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:18.311814 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:18.311814 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:18.311814 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:18.311814 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:18.312752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:18.312752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:18.312752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:18.312752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:18.312752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:18.312752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:18.312752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:18.312752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:18.312752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:18.312752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:18.312752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:29:18.312752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:18.312752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 226/448] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:29:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:18.317595 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdaa5b7-f6ba-4660-985b-96e1d15ef3dd None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:18.325222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f628d06-1d00-4e2c-93c5-c6b2833e2367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:18.326082 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f628d06-1d00-4e2c-93c5-c6b2833e2367 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:18.326364 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f628d06-1d00-4e2c-93c5-c6b2833e2367 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:18.326637 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f628d06-1d00-4e2c-93c5-c6b2833e2367 None admin] PATH_INFO: `/v3/groups/42a71e0d44eb4e3b8d35e0c3b45e9bc2` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:18.328444 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f628d06-1d00-4e2c-93c5-c6b2833e2367 None admin] RBAC: Authorizing `identity:get_group(group_id=42a71e0d44eb4e3b8d35e0c3b45e9bc2)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:18.330693 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f628d06-1d00-4e2c-93c5-c6b2833e2367 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:18.330957 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 223/449] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Jun 15 14:29:18 2022] GET /identity/v3/groups/42a71e0d44eb4e3b8d35e0c3b45e9bc2 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:18.338114 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f628d06-1d00-4e2c-93c5-c6b2833e2367 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:18.356942 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:18.430580 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdaa5b7-f6ba-4660-985b-96e1d15ef3dd None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:18.446664 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f628d06-1d00-4e2c-93c5-c6b2833e2367 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:18.462833 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982574b6-9f7f-49d5-b584-654659e3a637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:18.463524 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982574b6-9f7f-49d5-b584-654659e3a637 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:18.463804 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982574b6-9f7f-49d5-b584-654659e3a637 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:18.464611 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982574b6-9f7f-49d5-b584-654659e3a637 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:18.465426 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982574b6-9f7f-49d5-b584-654659e3a637 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:18.465735 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982574b6-9f7f-49d5-b584-654659e3a637 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:18.467147 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982574b6-9f7f-49d5-b584-654659e3a637 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:18.475004 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 227/450] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:18.480554 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6da45-9fe3-47aa-8726-b9d16a6457a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:18.481064 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982574b6-9f7f-49d5-b584-654659e3a637 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:18.481190 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6da45-9fe3-47aa-8726-b9d16a6457a6 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:18.481473 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6da45-9fe3-47aa-8726-b9d16a6457a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:18.481748 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6da45-9fe3-47aa-8726-b9d16a6457a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:18.482566 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c6da45-9fe3-47aa-8726-b9d16a6457a6 None admin] RBAC: Adding query filter params (group.id=42a71e0d44eb4e3b8d35e0c3b45e9bc2, role.id=ce0de9b9c9054826b569db895942ad26, scope.project.id=59af46f68a07464d897b0c71ece3f2f4) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:18.482873 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c6da45-9fe3-47aa-8726-b9d16a6457a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:18.485278 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c6da45-9fe3-47aa-8726-b9d16a6457a6 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:18.492014 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 224/451] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Jun 15 14:29:18 2022] GET /identity/v3/role_assignments?group.id=42a71e0d44eb4e3b8d35e0c3b45e9bc2&scope.project.id=59af46f68a07464d897b0c71ece3f2f4&role.id=ce0de9b9c9054826b569db895942ad26 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:18.581270 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:18.588171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982574b6-9f7f-49d5-b584-654659e3a637 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:18.624019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4ba7e8-1a21-4e66-a3b7-b16f4e5d02eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:18.624680 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ba7e8-1a21-4e66-a3b7-b16f4e5d02eb None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:18.624964 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ba7e8-1a21-4e66-a3b7-b16f4e5d02eb None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:18.625240 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ba7e8-1a21-4e66-a3b7-b16f4e5d02eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:18.694858 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4ba7e8-1a21-4e66-a3b7-b16f4e5d02eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:18.696369 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4ba7e8-1a21-4e66-a3b7-b16f4e5d02eb None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:18.790443 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:18.798019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 228/452] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 14:29:18.803656 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6da45-9fe3-47aa-8726-b9d16a6457a6 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:18.820564 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:18.909011 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6da45-9fe3-47aa-8726-b9d16a6457a6 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:18.943981 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7929c7-b5bb-4beb-8598-ebb20c628c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:18.944852 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7929c7-b5bb-4beb-8598-ebb20c628c91 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:18.945140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7929c7-b5bb-4beb-8598-ebb20c628c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:18.945414 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7929c7-b5bb-4beb-8598-ebb20c628c91 None admin] PATH_INFO: `/v3/projects/66e8228804bb44bd89c3894562c27b12` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:18.946599 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7929c7-b5bb-4beb-8598-ebb20c628c91 None admin] RBAC: Authorizing `identity:get_project(project_id=66e8228804bb44bd89c3894562c27b12)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:18.948061 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7929c7-b5bb-4beb-8598-ebb20c628c91 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:18.950256 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 225/453] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:18 2022] GET /identity/v3/projects/66e8228804bb44bd89c3894562c27b12 => generated 344 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:18.955766 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4ba7e8-1a21-4e66-a3b7-b16f4e5d02eb None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:19.051538 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4ba7e8-1a21-4e66-a3b7-b16f4e5d02eb None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:19.084164 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104452a4-8de0-405c-989c-27c2a95c286d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:19.085000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104452a4-8de0-405c-989c-27c2a95c286d None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:19.085277 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104452a4-8de0-405c-989c-27c2a95c286d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:19.085545 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104452a4-8de0-405c-989c-27c2a95c286d None admin] PATH_INFO: `/v3/groups/193167e463a748b8a7bdc11746687f39` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:19.086684 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104452a4-8de0-405c-989c-27c2a95c286d None admin] RBAC: Authorizing `identity:get_group(group_id=193167e463a748b8a7bdc11746687f39)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:19.088173 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104452a4-8de0-405c-989c-27c2a95c286d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:19.089804 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 229/454] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Jun 15 14:29:18 2022] GET /identity/v3/groups/193167e463a748b8a7bdc11746687f39 => generated 229 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:19.096637 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7929c7-b5bb-4beb-8598-ebb20c628c91 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:19.181289 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:19.200449 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7929c7-b5bb-4beb-8598-ebb20c628c91 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:19.231917 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2776f638-8903-4cb6-a435-e7277bc10e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:19.232549 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2776f638-8903-4cb6-a435-e7277bc10e16 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:19.232834 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2776f638-8903-4cb6-a435-e7277bc10e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:19.233108 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2776f638-8903-4cb6-a435-e7277bc10e16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:19.233929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2776f638-8903-4cb6-a435-e7277bc10e16 None admin] RBAC: Adding query filter params (group.id=193167e463a748b8a7bdc11746687f39, role.id=ed1cfa2fcf724a4d84f5acea100930c7, scope.project.id=66e8228804bb44bd89c3894562c27b12) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:19.234243 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2776f638-8903-4cb6-a435-e7277bc10e16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:19.235697 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2776f638-8903-4cb6-a435-e7277bc10e16 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:19.241316 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 226/455] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Jun 15 14:29:19 2022] GET /identity/v3/role_assignments?group.id=193167e463a748b8a7bdc11746687f39&scope.project.id=66e8228804bb44bd89c3894562c27b12&role.id=ed1cfa2fcf724a4d84f5acea100930c7 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:21.799111 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79f558-682e-4255-8d7b-ee065d5662db None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:21.800060 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79f558-682e-4255-8d7b-ee065d5662db None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:21.800570 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79f558-682e-4255-8d7b-ee065d5662db None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:21.802455 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 230/456] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:21.807411 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20247597-f678-4585-8ded-b6e716399159 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:21.808623 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20247597-f678-4585-8ded-b6e716399159 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:21.809113 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20247597-f678-4585-8ded-b6e716399159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:21.836086 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-20247597-f678-4585-8ded-b6e716399159 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:21.837296 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-20247597-f678-4585-8ded-b6e716399159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:21.924707 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 227/457] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:21.961202 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b79f558-682e-4255-8d7b-ee065d5662db None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:22.020026 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f88273-6cbc-4058-98fc-f1d96bdca759 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:22.020234 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f88273-6cbc-4058-98fc-f1d96bdca759 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:22.020719 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f88273-6cbc-4058-98fc-f1d96bdca759 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:22.022509 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 228/458] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:29:22.028727 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba3045-3dd1-4806-b618-e3d8993ddcbd None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:22.029211 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba3045-3dd1-4806-b618-e3d8993ddcbd None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:22.029659 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba3045-3dd1-4806-b618-e3d8993ddcbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:22.055443 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:22.063995 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-aaba3045-3dd1-4806-b618-e3d8993ddcbd None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:22.065192 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-aaba3045-3dd1-4806-b618-e3d8993ddcbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:22.116816 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b79f558-682e-4255-8d7b-ee065d5662db None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:22.165225 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75edeeb0-93ee-4a5e-b73f-aedc35c2421c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:22.166432 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edeeb0-93ee-4a5e-b73f-aedc35c2421c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:22.166913 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edeeb0-93ee-4a5e-b73f-aedc35c2421c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:22.167440 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edeeb0-93ee-4a5e-b73f-aedc35c2421c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:22.182781 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75edeeb0-93ee-4a5e-b73f-aedc35c2421c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:22.185304 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75edeeb0-93ee-4a5e-b73f-aedc35c2421c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:22.203533 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 229/459] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:22.213519 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-75edeeb0-93ee-4a5e-b73f-aedc35c2421c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:22.213519 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:22.213519 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:22.213519 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:22.213519 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:22.213519 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:22.213519 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:22.213519 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:22.213519 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:22.213519 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:22.213519 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:22.213519 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:22.213519 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:22.213519 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:22.213519 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:22.214382 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:22.214382 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:22.214382 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:22.214382 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:22.214382 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:22.214382 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:22.214382 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:22.214382 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:22.214382 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:22.214382 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:22.214382 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:22.214382 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:22.214382 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:22.214382 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:22.214382 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:22.214382 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:22.214382 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:22.215239 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:22.215239 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:22.215239 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:22.215239 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:22.217385 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 231/460] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:22.223274 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba3045-3dd1-4806-b618-e3d8993ddcbd None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:22.232396 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:22.243920 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e5c51-f220-402b-8c58-868c35770e4f None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:22.244208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e5c51-f220-402b-8c58-868c35770e4f None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:22.244490 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e5c51-f220-402b-8c58-868c35770e4f None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:22.247773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 232/461] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:22.252697 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aad5cf-0368-4a1d-adee-27c26396aa3f None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:22.253344 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aad5cf-0368-4a1d-adee-27c26396aa3f None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:22.253762 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aad5cf-0368-4a1d-adee-27c26396aa3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:22.277522 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-b9aad5cf-0368-4a1d-adee-27c26396aa3f None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:22.278934 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-b9aad5cf-0368-4a1d-adee-27c26396aa3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:22.359113 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 233/462] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:22.360252 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aad5cf-0368-4a1d-adee-27c26396aa3f None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:22.364032 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba3045-3dd1-4806-b618-e3d8993ddcbd None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:22.406614 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39368d-1e37-4717-94a7-680122a61153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:22.408115 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39368d-1e37-4717-94a7-680122a61153 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:22.408574 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39368d-1e37-4717-94a7-680122a61153 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:22.409004 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39368d-1e37-4717-94a7-680122a61153 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:22.410326 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f39368d-1e37-4717-94a7-680122a61153 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:22.411069 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f39368d-1e37-4717-94a7-680122a61153 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:22.412636 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f39368d-1e37-4717-94a7-680122a61153 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:22.428516 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:22.441072 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 230/463] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:22.443993 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39368d-1e37-4717-94a7-680122a61153 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:22.509435 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aad5cf-0368-4a1d-adee-27c26396aa3f None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:22.517502 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:22.545326 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4148c5-1194-43d1-9af2-e134b26fcbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:22.546124 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4148c5-1194-43d1-9af2-e134b26fcbfa None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:22.546407 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4148c5-1194-43d1-9af2-e134b26fcbfa None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:22.546734 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4148c5-1194-43d1-9af2-e134b26fcbfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:22.557151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4148c5-1194-43d1-9af2-e134b26fcbfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:22.557289 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39368d-1e37-4717-94a7-680122a61153 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:22.559633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4148c5-1194-43d1-9af2-e134b26fcbfa None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:22.579978 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-cf4148c5-1194-43d1-9af2-e134b26fcbfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:22.579978 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:22.579978 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:22.579978 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:22.579978 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:22.579978 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:22.579978 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:22.579978 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:22.579978 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:22.579978 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:22.579978 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:22.579978 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:22.579978 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:22.579978 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:22.579978 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:22.581045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:22.581045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:22.581045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:22.581045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:22.581045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:22.581045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:22.581045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:22.581045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:22.581045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:22.581045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:22.581045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:22.581045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:22.581045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:22.581045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:22.581045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:22.581045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:22.581045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:22.582011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:22.582011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:22.582011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:22.582011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:22.583877 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 234/464] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:22.584710 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4148c5-1194-43d1-9af2-e134b26fcbfa None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:22.597607 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77406d7f-fee8-4cf9-ba73-a561d0661996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:22.598395 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77406d7f-fee8-4cf9-ba73-a561d0661996 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:22.598676 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77406d7f-fee8-4cf9-ba73-a561d0661996 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:22.598948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77406d7f-fee8-4cf9-ba73-a561d0661996 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:22.609818 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77406d7f-fee8-4cf9-ba73-a561d0661996 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:22.611446 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77406d7f-fee8-4cf9-ba73-a561d0661996 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:22.629402 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-77406d7f-fee8-4cf9-ba73-a561d0661996 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:22.629402 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:22.629402 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:22.629402 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:22.629402 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:22.629402 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:22.629402 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:22.629402 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:22.629402 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:22.629402 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:22.629402 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:22.629402 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:22.629402 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:22.629402 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:22.629402 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:22.630246 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:22.630246 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:22.630246 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:22.630246 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:22.630246 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:22.630246 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:22.630246 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:22.630246 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:22.630246 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:22.630246 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:22.630246 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:22.630246 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:22.630246 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:22.630246 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:22.630246 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:22.630246 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:22.630246 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:22.636660 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:22.636660 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:22.636660 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:22.636660 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:22.636660 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 231/465] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:22.636660 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad21a4-b8d7-4b69-a7a2-b36c297ff0cb None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:22.636660 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad21a4-b8d7-4b69-a7a2-b36c297ff0cb None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:22.636660 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad21a4-b8d7-4b69-a7a2-b36c297ff0cb None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:22.636660 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 232/466] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:22.637134 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad21a4-b8d7-4b69-a7a2-b36c297ff0cb None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:22.696560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:22.700626 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4148c5-1194-43d1-9af2-e134b26fcbfa None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:22.746037 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc6bf6-c2fc-4b33-a20e-007a747d074a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:22.746725 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc6bf6-c2fc-4b33-a20e-007a747d074a None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:22.747030 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc6bf6-c2fc-4b33-a20e-007a747d074a None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:22.747125 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad21a4-b8d7-4b69-a7a2-b36c297ff0cb None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:22.747319 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc6bf6-c2fc-4b33-a20e-007a747d074a None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:22.748026 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc6bf6-c2fc-4b33-a20e-007a747d074a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:22.750185 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc6bf6-c2fc-4b33-a20e-007a747d074a None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:22.766357 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:22.770845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 235/467] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Jun 15 14:29:22 2022] POST /identity/v3/users => generated 246 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:29:22.772122 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2106e-1557-4886-a04e-fda08814479a None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:22.772485 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2106e-1557-4886-a04e-fda08814479a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:22.772833 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2106e-1557-4886-a04e-fda08814479a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:22.791548 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64874b-5fa9-43e5-9ff9-64c4046f86c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:22.793298 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64874b-5fa9-43e5-9ff9-64c4046f86c9 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:22.793799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64874b-5fa9-43e5-9ff9-64c4046f86c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:22.794290 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64874b-5fa9-43e5-9ff9-64c4046f86c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:22.795287 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64874b-5fa9-43e5-9ff9-64c4046f86c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:22.795853 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64874b-5fa9-43e5-9ff9-64c4046f86c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:22.797291 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-99c2106e-1557-4886-a04e-fda08814479a None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:22.797723 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64874b-5fa9-43e5-9ff9-64c4046f86c9 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:22.798616 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-99c2106e-1557-4886-a04e-fda08814479a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:22.812744 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 233/468] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:22.814259 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64874b-5fa9-43e5-9ff9-64c4046f86c9 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:22.887018 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 236/469] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:22.887018 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c2106e-1557-4886-a04e-fda08814479a None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:22.959186 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64874b-5fa9-43e5-9ff9-64c4046f86c9 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:22.978916 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:23.005752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c2106e-1557-4886-a04e-fda08814479a None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:23.011790 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09250ca-8cd9-4367-a06b-99a187d36ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:23.012935 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09250ca-8cd9-4367-a06b-99a187d36ced None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:23.013448 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09250ca-8cd9-4367-a06b-99a187d36ced None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:23.013961 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09250ca-8cd9-4367-a06b-99a187d36ced None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:23.015420 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09250ca-8cd9-4367-a06b-99a187d36ced None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:23.015738 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09250ca-8cd9-4367-a06b-99a187d36ced None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:23.017151 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09250ca-8cd9-4367-a06b-99a187d36ced None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:23.035128 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 234/470] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:23.036034 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09250ca-8cd9-4367-a06b-99a187d36ced None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:23.050423 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783d61d8-3e8b-4759-8d5e-30656ddbd4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:23.051266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d61d8-3e8b-4759-8d5e-30656ddbd4e9 None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:23.051556 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d61d8-3e8b-4759-8d5e-30656ddbd4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:23.051835 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d61d8-3e8b-4759-8d5e-30656ddbd4e9 None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:23.052549 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783d61d8-3e8b-4759-8d5e-30656ddbd4e9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:23.054019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783d61d8-3e8b-4759-8d5e-30656ddbd4e9 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:23.079627 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:23.092793 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 237/471] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Jun 15 14:29:22 2022] POST /identity/v3/users => generated 248 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:29:23.093685 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783d61d8-3e8b-4759-8d5e-30656ddbd4e9 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:23.162879 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09250ca-8cd9-4367-a06b-99a187d36ced None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:23.204249 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06560d2b-372c-4898-b5cd-70b3ed63f8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:23.205746 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06560d2b-372c-4898-b5cd-70b3ed63f8cc None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:23.206262 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06560d2b-372c-4898-b5cd-70b3ed63f8cc None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:23.207101 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06560d2b-372c-4898-b5cd-70b3ed63f8cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:23.223970 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06560d2b-372c-4898-b5cd-70b3ed63f8cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:23.226730 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06560d2b-372c-4898-b5cd-70b3ed63f8cc None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:23.228437 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783d61d8-3e8b-4759-8d5e-30656ddbd4e9 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:23.252323 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-06560d2b-372c-4898-b5cd-70b3ed63f8cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:23.252323 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:23.252323 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:23.252323 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:23.252323 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:23.252323 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:23.252323 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:23.252323 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:23.252323 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:23.252323 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:23.252323 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:23.252323 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:23.252323 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:23.252323 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:23.252323 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:23.253236 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:23.253236 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:23.253236 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:23.253236 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:23.253236 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:23.253236 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:23.253236 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:23.253236 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:23.253236 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:23.253236 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:23.253236 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:23.253236 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:23.253236 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:23.253236 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:23.253236 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:23.253236 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:23.253236 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:23.254071 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:23.254071 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:23.254071 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:23.254071 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:23.256668 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 235/472] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:23.262127 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06560d2b-372c-4898-b5cd-70b3ed63f8cc None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:23.286357 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14121bd5-1b77-403d-ac26-cb4c1424c326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:23.287501 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14121bd5-1b77-403d-ac26-cb4c1424c326 None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:23.287978 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14121bd5-1b77-403d-ac26-cb4c1424c326 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:23.288473 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14121bd5-1b77-403d-ac26-cb4c1424c326 None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:23.289393 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14121bd5-1b77-403d-ac26-cb4c1424c326 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:23.291198 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14121bd5-1b77-403d-ac26-cb4c1424c326 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:23.303591 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:23.340166 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 238/473] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Jun 15 14:29:23 2022] POST /identity/v3/users => generated 248 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:29:23.386384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376babd0-7443-4663-9dff-f599e4f7fe70 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:23.387092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376babd0-7443-4663-9dff-f599e4f7fe70 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:23.387578 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376babd0-7443-4663-9dff-f599e4f7fe70 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:23.390267 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 239/474] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:23.398150 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d71f9-aee7-45e1-aeb9-1469edfb314e None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:23.398774 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d71f9-aee7-45e1-aeb9-1469edfb314e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:23.399280 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d71f9-aee7-45e1-aeb9-1469edfb314e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:23.421774 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:23.425149 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-f58d71f9-aee7-45e1-aeb9-1469edfb314e None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:23.426805 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f58d71f9-aee7-45e1-aeb9-1469edfb314e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:23.437951 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06560d2b-372c-4898-b5cd-70b3ed63f8cc None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:23.489791 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c2957-cbcc-4400-976b-168e64a42f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:23.490542 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c2957-cbcc-4400-976b-168e64a42f40 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:23.490942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c2957-cbcc-4400-976b-168e64a42f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:23.491883 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c2957-cbcc-4400-976b-168e64a42f40 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:23.492670 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c2957-cbcc-4400-976b-168e64a42f40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:23.492979 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c2957-cbcc-4400-976b-168e64a42f40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:23.494701 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c2957-cbcc-4400-976b-168e64a42f40 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:23.513893 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 236/475] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:23.519619 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c2957-cbcc-4400-976b-168e64a42f40 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:23.527280 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 240/476] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:29:23.549157 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d71f9-aee7-45e1-aeb9-1469edfb314e None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:23.648345 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c2957-cbcc-4400-976b-168e64a42f40 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:23.684347 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d71f9-aee7-45e1-aeb9-1469edfb314e None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:23.687233 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf61ec6-699a-4826-967c-26d056974f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:23.687913 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf61ec6-699a-4826-967c-26d056974f48 None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:23.688200 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf61ec6-699a-4826-967c-26d056974f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:23.688479 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf61ec6-699a-4826-967c-26d056974f48 None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:23.689193 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf61ec6-699a-4826-967c-26d056974f48 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:23.690660 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf61ec6-699a-4826-967c-26d056974f48 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:23.714924 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:23.715882 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 237/477] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Jun 15 14:29:23 2022] POST /identity/v3/users => generated 249 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:29:23.733327 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c4e71-1263-4819-910b-c0d004e235af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:23.734035 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c4e71-1263-4819-910b-c0d004e235af None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:23.734317 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c4e71-1263-4819-910b-c0d004e235af None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:23.734589 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c4e71-1263-4819-910b-c0d004e235af None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:23.735473 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c4e71-1263-4819-910b-c0d004e235af None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:23.737034 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c4e71-1263-4819-910b-c0d004e235af None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:23.746734 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.sql.core [None req-517c4e71-1263-4819-910b-c0d004e235af None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 15 14:29:23.746734 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 15 14:29:23.746734 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [parameters: {'id': '9ab1e2fdfaac44eea036d97ff9eab560', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 15 14:29:23.746734 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80481) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 15 14:29:23.763424 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-517c4e71-1263-4819-910b-c0d004e235af None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 15 14:29:23.763424 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:23.763424 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 14:29:23.763424 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 14:29:23.763424 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 14:29:23.763424 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 14:29:23.763424 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 14:29:23.763424 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 14:29:23.763424 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 14:29:23.763424 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application conn.query(q) Jun 15 14:29:23.763424 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 14:29:23.763424 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 14:29:23.763424 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 14:29:23.763424 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application result.read() Jun 15 14:29:23.763424 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 14:29:23.764332 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 14:29:23.764332 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 14:29:23.764332 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 14:29:23.764332 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 14:29:23.764332 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 14:29:23.764332 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 14:29:23.764332 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 14:29:23.764332 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 15 14:29:23.764332 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:23.764332 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 15 14:29:23.764332 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:23.764332 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:23.764332 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 15 14:29:23.764332 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 15 14:29:23.764332 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 15 14:29:23.764332 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return ref.to_dict() Jun 15 14:29:23.764332 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 14:29:23.764332 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application next(self.gen) Jun 15 14:29:23.765561 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 15 14:29:23.765561 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application yield resource Jun 15 14:29:23.765561 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 14:29:23.765561 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application next(self.gen) Jun 15 14:29:23.765561 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 15 14:29:23.765561 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self.session.rollback() Jun 15 14:29:23.765561 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 15 14:29:23.765561 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self.force_reraise() Jun 15 14:29:23.765561 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 15 14:29:23.765561 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise self.value Jun 15 14:29:23.765561 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 15 14:29:23.765561 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 15 14:29:23.765561 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 15 14:29:23.765561 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application session.commit() Jun 15 14:29:23.765561 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 15 14:29:23.765561 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 15 14:29:23.765561 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 15 14:29:23.766418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._prepare_impl() Jun 15 14:29:23.766418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 15 14:29:23.766418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self.session.flush() Jun 15 14:29:23.766418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 15 14:29:23.766418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._flush(objects) Jun 15 14:29:23.766418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 15 14:29:23.766418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 15 14:29:23.766418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 15 14:29:23.766418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application compat.raise_( Jun 15 14:29:23.766418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 14:29:23.766418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception Jun 15 14:29:23.766418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 15 14:29:23.766418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application flush_context.execute() Jun 15 14:29:23.766418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 15 14:29:23.766418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rec.execute(self) Jun 15 14:29:23.766418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 15 14:29:23.766418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 15 14:29:23.767308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 15 14:29:23.767308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 15 14:29:23.767308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 15 14:29:23.767308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 15 14:29:23.767308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 15 14:29:23.767308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 15 14:29:23.767308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 15 14:29:23.767308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 15 14:29:23.767308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 15 14:29:23.767308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 15 14:29:23.767308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 15 14:29:23.767308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 15 14:29:23.767308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 15 14:29:23.767308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 15 14:29:23.767308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 14:29:23.767308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception Jun 15 14:29:23.768190 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 14:29:23.768190 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 14:29:23.768190 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 14:29:23.768190 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 14:29:23.768190 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 14:29:23.768190 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 14:29:23.768190 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 14:29:23.768190 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application conn.query(q) Jun 15 14:29:23.768190 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 14:29:23.768190 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 14:29:23.768190 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 14:29:23.768190 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application result.read() Jun 15 14:29:23.768190 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 14:29:23.768190 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 14:29:23.768190 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 14:29:23.768190 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 14:29:23.769231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 14:29:23.769231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 14:29:23.769231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 14:29:23.769231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 14:29:23.769231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 15 14:29:23.769231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 15 14:29:23.769231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [parameters: {'id': '9ab1e2fdfaac44eea036d97ff9eab560', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 15 14:29:23.769231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 15 14:29:23.769231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:23.769231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:29:23.769231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:23.769231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:23.769231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:23.769231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:23.769231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:23.769231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:23.770062 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:23.770062 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:23.770062 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:23.770062 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:23.770062 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:23.770062 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:23.770062 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 15 14:29:23.770062 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 15 14:29:23.770062 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:23.770062 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:23.770062 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 15 14:29:23.770062 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 15 14:29:23.770062 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 15 14:29:23.770062 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 15 14:29:23.770062 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 15 14:29:23.770062 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:23.770862 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 241/478] 10.222.0.45 () {60 vars in 1131 bytes} [Wed Jun 15 14:29:23 2022] POST /identity/v3/roles => generated 144 bytes in 216 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:29:23.774232 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf61ec6-699a-4826-967c-26d056974f48 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:23.784824 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:23.902560 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf61ec6-699a-4826-967c-26d056974f48 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:23.943144 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014207e-c7c6-462a-8d45-74b2f4b6e4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:23.943972 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014207e-c7c6-462a-8d45-74b2f4b6e4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:23.944259 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014207e-c7c6-462a-8d45-74b2f4b6e4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:23.944537 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014207e-c7c6-462a-8d45-74b2f4b6e4c7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:23.956018 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e014207e-c7c6-462a-8d45-74b2f4b6e4c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:23.964909 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-e014207e-c7c6-462a-8d45-74b2f4b6e4c7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:23.964909 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:23.964909 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:23.964909 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:23.964909 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:23.964909 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:23.964909 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:23.964909 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:23.964909 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:23.964909 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:23.964909 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:23.964909 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:23.964909 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:23.964909 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:23.964909 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:23.964909 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:23.967095 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:23.967095 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:23.967095 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:23.967095 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:23.967095 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:23.967095 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:23.967095 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:23.967095 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:23.967095 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:23.967095 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:23.967095 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:23.967095 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:23.967095 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:23.967095 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:23.967095 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:23.967095 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:23.969002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:23.969002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:23.969002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:23.969002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:23.969002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:23.969002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:23.969002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:23.969002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:23.969002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:23.969002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:23.969002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:23.969002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:23.969002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 238/479] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Jun 15 14:29:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:23.974615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c4e71-1263-4819-910b-c0d004e235af None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:24.075630 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c4e71-1263-4819-910b-c0d004e235af None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:24.117594 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8658f3b0-18be-4ad0-a463-1c7cee444171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:24.118278 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658f3b0-18be-4ad0-a463-1c7cee444171 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:24.118562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658f3b0-18be-4ad0-a463-1c7cee444171 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:24.118837 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658f3b0-18be-4ad0-a463-1c7cee444171 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:24.119625 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8658f3b0-18be-4ad0-a463-1c7cee444171 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:24.119936 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8658f3b0-18be-4ad0-a463-1c7cee444171 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:24.121404 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8658f3b0-18be-4ad0-a463-1c7cee444171 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:24.130460 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 242/480] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:25.371702 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94acc111-3531-445c-84d6-e4d51f863ce8 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:25.372208 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94acc111-3531-445c-84d6-e4d51f863ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:25.372307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94acc111-3531-445c-84d6-e4d51f863ce8 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:25.373889 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 239/481] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:25.388582 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682773d9-01fd-4829-b724-fb2687259e2d None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:25.389577 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682773d9-01fd-4829-b724-fb2687259e2d None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:25.389888 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682773d9-01fd-4829-b724-fb2687259e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:25.420347 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-682773d9-01fd-4829-b724-fb2687259e2d None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:25.421758 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-682773d9-01fd-4829-b724-fb2687259e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:25.480421 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:25.508036 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 243/482] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:25.543760 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94acc111-3531-445c-84d6-e4d51f863ce8 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:25.646909 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:25.694182 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94acc111-3531-445c-84d6-e4d51f863ce8 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:25.749685 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa5fc84-f3f1-4dd8-a0ca-54ac7c85df1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:25.750487 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5fc84-f3f1-4dd8-a0ca-54ac7c85df1c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:25.750776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5fc84-f3f1-4dd8-a0ca-54ac7c85df1c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:25.751094 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5fc84-f3f1-4dd8-a0ca-54ac7c85df1c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:25.761910 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa5fc84-f3f1-4dd8-a0ca-54ac7c85df1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:25.769889 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-cfa5fc84-f3f1-4dd8-a0ca-54ac7c85df1c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:25.769889 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:25.769889 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:25.769889 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:25.769889 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:25.769889 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:25.769889 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:25.769889 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:25.769889 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:25.769889 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:25.769889 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:25.769889 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:25.769889 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:25.769889 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:25.769889 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:25.769889 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:25.770748 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:25.770748 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:25.770748 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:25.770748 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:25.770748 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:25.770748 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:25.770748 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:25.770748 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:25.770748 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:25.770748 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:25.770748 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:25.770748 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:25.770748 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:25.770748 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:25.770748 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:25.770748 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:25.771544 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:25.771544 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:25.771544 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:25.771544 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:25.771544 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:25.771544 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:25.771544 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:25.771544 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:25.771544 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:25.771544 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:25.771544 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:25.771544 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:25.771544 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 240/483] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Jun 15 14:29:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:25.777302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682773d9-01fd-4829-b724-fb2687259e2d None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:25.903887 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4c7a0-ffa8-4a99-9a6e-158c3ec04f32 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:25.904485 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4c7a0-ffa8-4a99-9a6e-158c3ec04f32 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:25.904961 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4c7a0-ffa8-4a99-9a6e-158c3ec04f32 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:25.908054 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 241/484] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:25.912423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee542634-9d75-42fe-aab7-e9a0bd1102f8 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:25.912703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee542634-9d75-42fe-aab7-e9a0bd1102f8 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:25.912977 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee542634-9d75-42fe-aab7-e9a0bd1102f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:25.933484 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682773d9-01fd-4829-b724-fb2687259e2d None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:25.952209 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-ee542634-9d75-42fe-aab7-e9a0bd1102f8 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:25.953464 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ee542634-9d75-42fe-aab7-e9a0bd1102f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:25.971900 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b9ed2f-0f1c-4f50-acb1-e2049f7c8103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:25.972583 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9ed2f-0f1c-4f50-acb1-e2049f7c8103 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:25.972872 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9ed2f-0f1c-4f50-acb1-e2049f7c8103 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:25.973154 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9ed2f-0f1c-4f50-acb1-e2049f7c8103 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:25.973910 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b9ed2f-0f1c-4f50-acb1-e2049f7c8103 None admin] RBAC: Adding query filter params (name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:25.974221 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b9ed2f-0f1c-4f50-acb1-e2049f7c8103 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:25.975719 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b9ed2f-0f1c-4f50-acb1-e2049f7c8103 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:25.996439 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 244/485] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Jun 15 14:29:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:26.003985 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b9ed2f-0f1c-4f50-acb1-e2049f7c8103 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:26.064603 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 242/486] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:26.083414 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:26.101689 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee542634-9d75-42fe-aab7-e9a0bd1102f8 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:26.179913 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b9ed2f-0f1c-4f50-acb1-e2049f7c8103 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:26.184588 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:26.252006 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27791e0d-d4b4-4cbd-be6f-556c372084c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:26.252806 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27791e0d-d4b4-4cbd-be6f-556c372084c7 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:26.253086 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27791e0d-d4b4-4cbd-be6f-556c372084c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:26.253357 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27791e0d-d4b4-4cbd-be6f-556c372084c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:26.267117 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27791e0d-d4b4-4cbd-be6f-556c372084c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:26.267579 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee542634-9d75-42fe-aab7-e9a0bd1102f8 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:26.269427 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27791e0d-d4b4-4cbd-be6f-556c372084c7 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:26.288890 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-27791e0d-d4b4-4cbd-be6f-556c372084c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:26.288890 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:26.288890 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:26.288890 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:26.288890 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:26.288890 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:26.288890 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:26.288890 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:26.288890 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:26.288890 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:26.288890 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:26.288890 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:26.288890 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:26.288890 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:26.288890 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:26.289785 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:26.289785 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:26.289785 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:26.289785 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:26.289785 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:26.289785 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:26.289785 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:26.289785 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:26.289785 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:26.289785 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:26.289785 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:26.289785 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:26.289785 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:26.289785 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:26.289785 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:26.289785 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:26.289785 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:26.290610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:26.290610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:26.290610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:26.290610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:26.292940 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 245/487] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:26.294408 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a54f9-a059-4dcd-a4e0-329e64b1b6d7 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:26.294893 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a54f9-a059-4dcd-a4e0-329e64b1b6d7 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:26.295388 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a54f9-a059-4dcd-a4e0-329e64b1b6d7 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:26.296997 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 246/488] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:26.301949 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a54f9-a059-4dcd-a4e0-329e64b1b6d7 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:26.304807 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0f3164-6d2e-41e6-b6a1-937266332d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:26.305716 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f3164-6d2e-41e6-b6a1-937266332d74 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:26.306003 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f3164-6d2e-41e6-b6a1-937266332d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:26.306278 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f3164-6d2e-41e6-b6a1-937266332d74 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:26.317354 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0f3164-6d2e-41e6-b6a1-937266332d74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:26.326002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-de0f3164-6d2e-41e6-b6a1-937266332d74 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:26.326002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:26.326002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:26.326002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:26.326002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:26.326002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:26.326002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:26.326002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:26.326002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:26.326002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:26.326002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:26.326002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:26.326002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:26.326002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:26.326002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:26.326002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:26.326845 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:26.326845 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:26.326845 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:26.326845 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:26.326845 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:26.326845 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:26.326845 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:26.326845 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:26.326845 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:26.326845 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:26.326845 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:26.326845 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:26.326845 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:26.326845 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:26.326845 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:26.326845 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:26.327939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:26.327939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:26.327939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:26.327939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:26.327939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:26.327939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:26.327939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:26.327939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:26.327939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:26.327939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:26.327939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:26.327939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:26.327939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 243/489] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Jun 15 14:29:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:29:26.333470 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22782aa7-276e-4cb2-a02d-22103e807904 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:26.333756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22782aa7-276e-4cb2-a02d-22103e807904 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:26.334029 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22782aa7-276e-4cb2-a02d-22103e807904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:26.352139 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-22782aa7-276e-4cb2-a02d-22103e807904 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:26.353369 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-22782aa7-276e-4cb2-a02d-22103e807904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:26.412793 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:26.421329 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a54f9-a059-4dcd-a4e0-329e64b1b6d7 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:26.433351 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 244/490] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:26.435458 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22782aa7-276e-4cb2-a02d-22103e807904 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:26.479089 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85afc562-3c75-4294-bac7-b656d8126cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:26.479424 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85afc562-3c75-4294-bac7-b656d8126cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:26.479608 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85afc562-3c75-4294-bac7-b656d8126cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:26.481905 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85afc562-3c75-4294-bac7-b656d8126cd0 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:26.481905 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85afc562-3c75-4294-bac7-b656d8126cd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:26.481905 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85afc562-3c75-4294-bac7-b656d8126cd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:26.482419 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85afc562-3c75-4294-bac7-b656d8126cd0 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:26.486631 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:26.496127 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 247/491] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:26.497418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded78c1f-00e0-44d3-a6b5-76fa4f2f0b5a None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:26.497704 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded78c1f-00e0-44d3-a6b5-76fa4f2f0b5a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:26.497979 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded78c1f-00e0-44d3-a6b5-76fa4f2f0b5a None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:26.499353 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 248/492] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:26.499898 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded78c1f-00e0-44d3-a6b5-76fa4f2f0b5a None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:26.549102 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22782aa7-276e-4cb2-a02d-22103e807904 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:26.586795 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168cf4e-c215-465b-ad6f-1fa8e5aaa32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:26.587505 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168cf4e-c215-465b-ad6f-1fa8e5aaa32b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:26.587812 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168cf4e-c215-465b-ad6f-1fa8e5aaa32b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:26.588064 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168cf4e-c215-465b-ad6f-1fa8e5aaa32b None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:26.588812 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f168cf4e-c215-465b-ad6f-1fa8e5aaa32b None admin] RBAC: Adding query filter params (name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:26.589118 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f168cf4e-c215-465b-ad6f-1fa8e5aaa32b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:26.590521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f168cf4e-c215-465b-ad6f-1fa8e5aaa32b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:26.603043 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 245/493] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Jun 15 14:29:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:26.603043 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168cf4e-c215-465b-ad6f-1fa8e5aaa32b None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:26.612860 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded78c1f-00e0-44d3-a6b5-76fa4f2f0b5a None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:26.648878 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2e65c-fad3-4aa5-b070-d72096ea84f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:26.649677 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2e65c-fad3-4aa5-b070-d72096ea84f5 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:26.650012 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2e65c-fad3-4aa5-b070-d72096ea84f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:26.650290 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2e65c-fad3-4aa5-b070-d72096ea84f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:26.660449 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac2e65c-fad3-4aa5-b070-d72096ea84f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:26.668063 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-8ac2e65c-fad3-4aa5-b070-d72096ea84f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:26.668063 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:26.668063 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:26.668063 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:26.668063 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:26.668063 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:26.668063 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:26.668063 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:26.668063 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:26.668063 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:26.668063 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:26.668063 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:26.668063 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:26.668063 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:26.668063 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:26.668063 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:26.669013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:26.669013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:26.669013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:26.669013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:26.669013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:26.669013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:26.669013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:26.669013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:26.669013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:26.669013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:26.669013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:26.669013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:26.669013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:26.669013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:26.669013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:26.669013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:26.669867 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:26.669867 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:26.669867 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:26.669867 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:26.669867 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:26.669867 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:26.669867 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:26.669867 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:26.669867 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:26.669867 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:26.669867 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:26.669867 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:26.669867 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 249/494] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Jun 15 14:29:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:26.670531 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb0e8c-c169-4f92-bd08-d0e667f17fdb None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:26.671093 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb0e8c-c169-4f92-bd08-d0e667f17fdb None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:26.671093 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb0e8c-c169-4f92-bd08-d0e667f17fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:26.687876 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-44cb0e8c-c169-4f92-bd08-d0e667f17fdb None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:26.688987 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-44cb0e8c-c169-4f92-bd08-d0e667f17fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:26.700516 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:26.716971 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168cf4e-c215-465b-ad6f-1fa8e5aaa32b None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:26.730785 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:26.760783 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f076b014-f2bd-4b0e-9073-0f68cee3492f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:26.762143 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f076b014-f2bd-4b0e-9073-0f68cee3492f None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:26.762427 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f076b014-f2bd-4b0e-9073-0f68cee3492f None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:26.762702 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f076b014-f2bd-4b0e-9073-0f68cee3492f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:26.765763 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 250/495] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:26.766610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb0e8c-c169-4f92-bd08-d0e667f17fdb None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:26.780815 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f076b014-f2bd-4b0e-9073-0f68cee3492f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:26.782441 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f076b014-f2bd-4b0e-9073-0f68cee3492f None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:26.803159 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-f076b014-f2bd-4b0e-9073-0f68cee3492f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:29:26.803159 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:26.803159 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:26.803159 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:26.803159 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:26.803159 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:26.803159 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:26.803159 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:26.803159 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:26.803159 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:26.803159 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:26.803159 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:26.803159 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:29:26.803159 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:29:26.803159 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:29:26.803159 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:29:26.804058 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:26.804058 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:26.804058 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:29:26.804058 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:26.804058 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:29:26.804058 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:26.804058 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:26.804058 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:26.804058 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:26.804058 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:26.804058 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:26.804058 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:26.804058 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:26.804058 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:26.804058 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:26.804058 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:26.804058 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:26.804058 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:26.805571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:26.805571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:26.805571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:29:26.805571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:29:26.805571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:29:26.805571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:29:26.805571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:29:26.805571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:29:26.805571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:29:26.805571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:26.805571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 246/496] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Jun 15 14:29:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:26.806185 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f076b014-f2bd-4b0e-9073-0f68cee3492f None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:26.873069 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb0e8c-c169-4f92-bd08-d0e667f17fdb None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:26.908844 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168006a0-0871-4cf5-8ed3-9ea955810cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:26.909635 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168006a0-0871-4cf5-8ed3-9ea955810cd7 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:26.909920 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168006a0-0871-4cf5-8ed3-9ea955810cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:26.910196 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168006a0-0871-4cf5-8ed3-9ea955810cd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:26.911246 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f076b014-f2bd-4b0e-9073-0f68cee3492f None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:26.920394 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168006a0-0871-4cf5-8ed3-9ea955810cd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:26.921843 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168006a0-0871-4cf5-8ed3-9ea955810cd7 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:26.928955 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:26.943288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-168006a0-0871-4cf5-8ed3-9ea955810cd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:26.943288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:26.943288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:26.943288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:26.943288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:26.943288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:26.943288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:26.943288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:26.943288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:26.943288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:26.943288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:26.943288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:26.943288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:26.943288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:26.943288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:26.944092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:26.944092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:26.944092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:26.944092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:26.944092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:26.944092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:26.944092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:26.944092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:26.944092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:26.944092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:26.944092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:26.944092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:26.944092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:26.944092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:26.944092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:26.944092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:26.944092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:26.944919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:26.944919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:26.944919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:26.944919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:26.944919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 251/497] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:26.945857 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168006a0-0871-4cf5-8ed3-9ea955810cd7 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:26.953852 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85e51d-4ef9-48b5-b2b8-65947f96f7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:26.954526 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85e51d-4ef9-48b5-b2b8-65947f96f7fe None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:26.954811 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85e51d-4ef9-48b5-b2b8-65947f96f7fe None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:26.955119 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85e51d-4ef9-48b5-b2b8-65947f96f7fe None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:26.955875 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb85e51d-4ef9-48b5-b2b8-65947f96f7fe None admin] RBAC: Adding query filter params (name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:26.956187 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb85e51d-4ef9-48b5-b2b8-65947f96f7fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:26.957574 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb85e51d-4ef9-48b5-b2b8-65947f96f7fe None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:26.971150 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 247/498] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Jun 15 14:29:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:26.972222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85e51d-4ef9-48b5-b2b8-65947f96f7fe None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:27.006839 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:27.063341 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168006a0-0871-4cf5-8ed3-9ea955810cd7 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:27.082889 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85e51d-4ef9-48b5-b2b8-65947f96f7fe None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:27.097822 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2f1759-1725-493d-8517-8b4da269b41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:27.098620 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f1759-1725-493d-8517-8b4da269b41e None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:27.098898 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f1759-1725-493d-8517-8b4da269b41e None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:27.099222 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f1759-1725-493d-8517-8b4da269b41e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:27.109196 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2f1759-1725-493d-8517-8b4da269b41e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:27.115786 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4aa475-f6e6-4156-b562-10655986eadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:27.116444 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4aa475-f6e6-4156-b562-10655986eadc None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:27.116727 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4aa475-f6e6-4156-b562-10655986eadc None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:27.117000 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4aa475-f6e6-4156-b562-10655986eadc None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:27.117088 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-cd2f1759-1725-493d-8517-8b4da269b41e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:27.117088 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:27.117088 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:27.117088 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:27.117088 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:27.117088 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:27.117088 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:27.117088 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:27.117088 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:27.117088 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:27.117088 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:27.117088 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:27.117088 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:27.117088 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:27.117088 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:27.117088 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:27.118099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:27.118099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:27.118099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:27.118099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:27.118099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:27.118099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:27.118099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:27.118099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:27.118099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:27.118099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:27.118099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:27.118099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:27.118099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:27.118099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:27.118099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:27.118099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:27.118895 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:27.118895 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:27.118895 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:27.118895 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:27.118895 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:27.118895 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:27.118895 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:27.118895 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:27.118895 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:27.118895 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:27.118895 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:27.118895 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:27.119534 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4aa475-f6e6-4156-b562-10655986eadc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:27.119534 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4aa475-f6e6-4156-b562-10655986eadc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:27.119667 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 252/499] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Jun 15 14:29:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:27.119667 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2f1759-1725-493d-8517-8b4da269b41e None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:27.119783 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4aa475-f6e6-4156-b562-10655986eadc None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:27.157265 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 248/500] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Jun 15 14:29:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:27.158004 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4aa475-f6e6-4156-b562-10655986eadc None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:27.180141 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:27.223073 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2f1759-1725-493d-8517-8b4da269b41e None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:27.245713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:27.256905 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b084c-5239-46a4-8df0-93505ab04800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:27.257564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b084c-5239-46a4-8df0-93505ab04800 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:27.257852 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b084c-5239-46a4-8df0-93505ab04800 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:27.258105 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4aa475-f6e6-4156-b562-10655986eadc None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:27.258195 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b084c-5239-46a4-8df0-93505ab04800 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:27.258840 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5b084c-5239-46a4-8df0-93505ab04800 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:27.259181 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5b084c-5239-46a4-8df0-93505ab04800 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:27.260577 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5b084c-5239-46a4-8df0-93505ab04800 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:27.272861 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 253/501] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:27.274241 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecdd64-745a-4466-90b7-06cb33b4699d None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:27.274660 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecdd64-745a-4466-90b7-06cb33b4699d None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:27.274986 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecdd64-745a-4466-90b7-06cb33b4699d None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:27.276405 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 254/502] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:27.277547 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daecdd64-745a-4466-90b7-06cb33b4699d None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:27.294217 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be8951-848d-4591-af41-83dd5506ed5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:27.295008 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be8951-848d-4591-af41-83dd5506ed5d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:27.295309 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be8951-848d-4591-af41-83dd5506ed5d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:27.295584 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be8951-848d-4591-af41-83dd5506ed5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:27.305839 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46be8951-848d-4591-af41-83dd5506ed5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:27.307327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46be8951-848d-4591-af41-83dd5506ed5d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:27.325423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-46be8951-848d-4591-af41-83dd5506ed5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:27.325423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:27.325423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:27.325423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:27.325423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:27.325423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:27.325423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:27.325423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:27.325423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:27.325423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:27.325423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:27.325423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:27.325423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:27.325423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:27.325423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:27.326418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:27.326418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:27.326418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:27.326418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:27.326418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:27.326418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:27.326418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:27.326418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:27.326418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:27.326418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:27.326418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:27.326418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:27.326418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:27.326418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:27.326418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:27.326418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:27.326418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:27.327264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:27.327264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:27.327264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:27.327264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:27.327264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 249/503] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:27.327536 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be8951-848d-4591-af41-83dd5506ed5d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:27.382236 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daecdd64-745a-4466-90b7-06cb33b4699d None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:27.400097 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:27.416656 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a06a980-392e-4e9b-866a-2b73b1754747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:27.417342 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06a980-392e-4e9b-866a-2b73b1754747 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:27.417630 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06a980-392e-4e9b-866a-2b73b1754747 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:27.417908 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06a980-392e-4e9b-866a-2b73b1754747 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:27.418665 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a06a980-392e-4e9b-866a-2b73b1754747 None admin] RBAC: Adding query filter params (name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:27.418975 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a06a980-392e-4e9b-866a-2b73b1754747 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:27.420434 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a06a980-392e-4e9b-866a-2b73b1754747 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:27.424090 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be8951-848d-4591-af41-83dd5506ed5d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:27.428329 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 255/504] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Jun 15 14:29:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:27.429108 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a06a980-392e-4e9b-866a-2b73b1754747 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:27.460383 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d13d4f7-5b56-41f3-8aa0-8fcd38925358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:27.461025 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13d4f7-5b56-41f3-8aa0-8fcd38925358 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:27.461306 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13d4f7-5b56-41f3-8aa0-8fcd38925358 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:27.461582 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13d4f7-5b56-41f3-8aa0-8fcd38925358 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:27.472024 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:27.529271 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d13d4f7-5b56-41f3-8aa0-8fcd38925358 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:27.529456 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a06a980-392e-4e9b-866a-2b73b1754747 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:27.530667 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d13d4f7-5b56-41f3-8aa0-8fcd38925358 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:27.562026 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec9e1b-be65-4981-b50f-48c346aceaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:27.562814 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec9e1b-be65-4981-b50f-48c346aceaa3 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:27.563125 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec9e1b-be65-4981-b50f-48c346aceaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:27.563478 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec9e1b-be65-4981-b50f-48c346aceaa3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:27.570808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec9e1b-be65-4981-b50f-48c346aceaa3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:27.572286 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec9e1b-be65-4981-b50f-48c346aceaa3 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:27.588068 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-7fec9e1b-be65-4981-b50f-48c346aceaa3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 15 14:29:27.588068 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:27.588068 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:27.588068 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:27.588068 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:27.588068 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:27.588068 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:27.588068 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:27.588068 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:27.588068 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:27.588068 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:27.588068 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:27.588068 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:29:27.588068 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:29:27.588068 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:29:27.588068 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:29:27.589000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:27.589000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:27.589000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:29:27.589000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:27.589000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:29:27.589000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:27.589000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:27.589000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:27.589000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:27.589000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:27.589000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:27.589000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:27.589000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:27.589000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:27.589000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:27.589000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:27.589000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:27.589000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:27.589943 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:27.589943 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:27.589943 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:29:27.589943 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:29:27.589943 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:29:27.589943 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:29:27.589943 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:29:27.589943 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:29:27.589943 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 15 14:29:27.589943 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:27.589943 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 256/505] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:27.590482 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d1d3a-09ad-48bf-818d-bd763dfa3d80 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:27.590829 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d1d3a-09ad-48bf-818d-bd763dfa3d80 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:27.591071 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d1d3a-09ad-48bf-818d-bd763dfa3d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:27.606889 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-0c3d1d3a-09ad-48bf-818d-bd763dfa3d80 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:27.608112 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-0c3d1d3a-09ad-48bf-818d-bd763dfa3d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:27.632278 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 250/506] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:27.633220 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d13d4f7-5b56-41f3-8aa0-8fcd38925358 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:27.646256 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:27.674114 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:27.679201 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 257/507] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:27.680035 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d1d3a-09ad-48bf-818d-bd763dfa3d80 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:27.741417 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d13d4f7-5b56-41f3-8aa0-8fcd38925358 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:27.779606 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d1d3a-09ad-48bf-818d-bd763dfa3d80 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:27.781558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59a617f-20cf-46e0-bcdf-0406522fdcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:27.782225 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a617f-20cf-46e0-bcdf-0406522fdcc5 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:27.782539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a617f-20cf-46e0-bcdf-0406522fdcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:27.782824 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a617f-20cf-46e0-bcdf-0406522fdcc5 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:27.783578 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59a617f-20cf-46e0-bcdf-0406522fdcc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:27.783889 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59a617f-20cf-46e0-bcdf-0406522fdcc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:27.785283 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59a617f-20cf-46e0-bcdf-0406522fdcc5 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:27.796968 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 251/508] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:27.797757 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59a617f-20cf-46e0-bcdf-0406522fdcc5 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:27.818468 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30badb41-66c9-403b-b3da-292d653981fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:27.819299 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30badb41-66c9-403b-b3da-292d653981fe None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:27.819585 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30badb41-66c9-403b-b3da-292d653981fe None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:27.819869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30badb41-66c9-403b-b3da-292d653981fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:27.829866 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30badb41-66c9-403b-b3da-292d653981fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:27.831383 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30badb41-66c9-403b-b3da-292d653981fe None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:27.848642 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-30badb41-66c9-403b-b3da-292d653981fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:27.848642 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:27.848642 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:27.848642 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:27.848642 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:27.848642 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:27.848642 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:27.848642 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:27.848642 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:27.848642 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:27.848642 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:27.848642 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:27.848642 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:27.848642 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:27.848642 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:27.849619 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:27.849619 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:27.849619 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:27.849619 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:27.849619 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:27.849619 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:27.849619 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:27.849619 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:27.849619 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:27.849619 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:27.849619 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:27.849619 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:27.849619 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:27.849619 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:27.849619 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:27.849619 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:27.849619 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:27.850496 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:27.850496 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:27.850496 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:27.850496 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:27.850496 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 258/509] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:27.850960 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30badb41-66c9-403b-b3da-292d653981fe None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:27.876095 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:27.893240 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:27.896345 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59a617f-20cf-46e0-bcdf-0406522fdcc5 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:27.929769 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281bfde3-30c5-47f2-9a37-a136c63684fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:27.930517 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281bfde3-30c5-47f2-9a37-a136c63684fa None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:27.930877 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281bfde3-30c5-47f2-9a37-a136c63684fa None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:27.931248 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281bfde3-30c5-47f2-9a37-a136c63684fa None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:27.932080 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281bfde3-30c5-47f2-9a37-a136c63684fa None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:27.932399 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281bfde3-30c5-47f2-9a37-a136c63684fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:27.934432 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281bfde3-30c5-47f2-9a37-a136c63684fa None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:27.950827 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30badb41-66c9-403b-b3da-292d653981fe None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:27.951817 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 252/510] 10.222.0.45 () {56 vars in 1137 bytes} [Wed Jun 15 14:29:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:27.952605 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281bfde3-30c5-47f2-9a37-a136c63684fa None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:27.988397 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f74f4c-fb90-43f2-83a4-1c49765d2144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:27.989177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f74f4c-fb90-43f2-83a4-1c49765d2144 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:27.989455 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f74f4c-fb90-43f2-83a4-1c49765d2144 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:27.989722 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f74f4c-fb90-43f2-83a4-1c49765d2144 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:28.000049 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f74f4c-fb90-43f2-83a4-1c49765d2144 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:28.001486 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f74f4c-fb90-43f2-83a4-1c49765d2144 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:28.018325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-55f74f4c-fb90-43f2-83a4-1c49765d2144 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:28.018325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:28.018325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:28.018325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:28.018325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:28.018325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:28.018325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:28.018325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:28.018325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:28.018325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:28.018325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:28.018325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:28.018325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:28.018325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:28.018325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:28.019628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:28.019628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:28.019628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:28.019628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:28.019628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:28.019628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:28.019628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:28.019628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:28.019628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:28.019628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:28.019628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:28.019628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:28.019628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:28.019628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:28.019628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:28.019628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:28.019628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:28.020688 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:28.020688 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:28.020688 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:28.020688 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:28.020688 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 259/511] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:28.020688 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f74f4c-fb90-43f2-83a4-1c49765d2144 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:28.057852 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281bfde3-30c5-47f2-9a37-a136c63684fa None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:28.091154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1649a2-b461-4a5a-9ea4-9c1f0a9db341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:28.091947 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1649a2-b461-4a5a-9ea4-9c1f0a9db341 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:28.092230 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1649a2-b461-4a5a-9ea4-9c1f0a9db341 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:28.092507 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1649a2-b461-4a5a-9ea4-9c1f0a9db341 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:28.096367 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:28.102595 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1649a2-b461-4a5a-9ea4-9c1f0a9db341 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:28.104101 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1649a2-b461-4a5a-9ea4-9c1f0a9db341 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:28.121668 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-7b1649a2-b461-4a5a-9ea4-9c1f0a9db341 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:28.121668 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:28.121668 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:28.121668 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:28.121668 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:28.121668 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:28.121668 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:28.121668 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:28.121668 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:28.121668 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:28.121668 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:28.121668 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:28.121668 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:28.121668 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:28.121668 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:28.122662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:28.122662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:28.122662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:28.122662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:28.122662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:28.122662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:28.122662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:28.122662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:28.122662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:28.122662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:28.122662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:28.122662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:28.122662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:28.122662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:28.122662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:28.122662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:28.122662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:28.123642 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:28.123642 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:28.123642 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:28.123642 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:28.124039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f74f4c-fb90-43f2-83a4-1c49765d2144 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:28.124130 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 253/512] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:28.124130 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1649a2-b461-4a5a-9ea4-9c1f0a9db341 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:28.136892 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:28.157271 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b04447-82b4-44f4-8caf-8a7033a39e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:28.158066 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b04447-82b4-44f4-8caf-8a7033a39e13 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:28.158614 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b04447-82b4-44f4-8caf-8a7033a39e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:28.158614 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b04447-82b4-44f4-8caf-8a7033a39e13 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:28.165570 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b04447-82b4-44f4-8caf-8a7033a39e13 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:28.167256 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b04447-82b4-44f4-8caf-8a7033a39e13 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:28.181670 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-a3b04447-82b4-44f4-8caf-8a7033a39e13 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 14:29:28.181670 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:28.181670 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:28.181670 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:28.181670 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:28.181670 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:28.181670 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:28.181670 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:28.181670 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:28.181670 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:28.181670 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:28.181670 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:28.181670 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:29:28.181670 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:29:28.181670 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:29:28.181670 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:29:28.182730 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:28.182730 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:28.182730 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:29:28.182730 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:28.182730 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:29:28.182730 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:28.182730 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:28.182730 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:28.182730 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:28.182730 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:28.182730 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:28.182730 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:28.182730 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:28.182730 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:28.182730 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:28.182730 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:28.182730 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:28.182730 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:28.183848 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:28.183848 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:28.183848 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:29:28.183848 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:29:28.183848 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:29:28.183848 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:29:28.183848 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:29:28.183848 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:29:28.183848 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 14:29:28.183848 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:28.183848 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 260/513] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:28.183848 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b04447-82b4-44f4-8caf-8a7033a39e13 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:28.227830 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1649a2-b461-4a5a-9ea4-9c1f0a9db341 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:28.265966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7bb419-52ee-482b-86a9-96b7f89f388a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:28.266640 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7bb419-52ee-482b-86a9-96b7f89f388a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:28.266924 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7bb419-52ee-482b-86a9-96b7f89f388a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:28.267235 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7bb419-52ee-482b-86a9-96b7f89f388a None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:28.267964 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7bb419-52ee-482b-86a9-96b7f89f388a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:28.268272 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7bb419-52ee-482b-86a9-96b7f89f388a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:28.269911 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7bb419-52ee-482b-86a9-96b7f89f388a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:28.281639 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 254/514] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:28.282427 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7bb419-52ee-482b-86a9-96b7f89f388a None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:28.288069 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b04447-82b4-44f4-8caf-8a7033a39e13 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:28.321648 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b8fdb-4f1f-4e09-a206-ef0a540eca7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:28.322225 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b8fdb-4f1f-4e09-a206-ef0a540eca7e None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:28.322505 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b8fdb-4f1f-4e09-a206-ef0a540eca7e None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:28.322782 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b8fdb-4f1f-4e09-a206-ef0a540eca7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:28.325673 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:28.351287 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:28.382303 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7bb419-52ee-482b-86a9-96b7f89f388a None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:28.390836 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7b8fdb-4f1f-4e09-a206-ef0a540eca7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:28.392336 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7b8fdb-4f1f-4e09-a206-ef0a540eca7e None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:28.416121 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe447b9a-6c2f-41fc-85c0-8864c41fe083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:28.416770 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe447b9a-6c2f-41fc-85c0-8864c41fe083 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:28.417051 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe447b9a-6c2f-41fc-85c0-8864c41fe083 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:28.417328 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe447b9a-6c2f-41fc-85c0-8864c41fe083 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:28.418038 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe447b9a-6c2f-41fc-85c0-8864c41fe083 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:28.418399 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe447b9a-6c2f-41fc-85c0-8864c41fe083 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:28.419828 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe447b9a-6c2f-41fc-85c0-8864c41fe083 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:28.431613 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 255/515] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:28.432401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe447b9a-6c2f-41fc-85c0-8864c41fe083 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:28.490417 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 261/516] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:28.491287 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b8fdb-4f1f-4e09-a206-ef0a540eca7e None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:28.525018 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:28.531315 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe447b9a-6c2f-41fc-85c0-8864c41fe083 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:28.559449 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:28.565889 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feabae2-4bea-498b-b39d-8f0038104238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:28.566538 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feabae2-4bea-498b-b39d-8f0038104238 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:28.566818 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feabae2-4bea-498b-b39d-8f0038104238 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:28.567129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feabae2-4bea-498b-b39d-8f0038104238 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:28.567854 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feabae2-4bea-498b-b39d-8f0038104238 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:28.568162 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feabae2-4bea-498b-b39d-8f0038104238 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:28.569613 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feabae2-4bea-498b-b39d-8f0038104238 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:28.581154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 256/517] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:28.581880 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feabae2-4bea-498b-b39d-8f0038104238 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:28.590808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b8fdb-4f1f-4e09-a206-ef0a540eca7e None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:28.624000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469dbde-3b57-49c7-b344-a9b3ef1d54e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:28.624662 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469dbde-3b57-49c7-b344-a9b3ef1d54e7 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:28.624943 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469dbde-3b57-49c7-b344-a9b3ef1d54e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:28.625214 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469dbde-3b57-49c7-b344-a9b3ef1d54e7 None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:28.626039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469dbde-3b57-49c7-b344-a9b3ef1d54e7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:28.626347 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469dbde-3b57-49c7-b344-a9b3ef1d54e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:28.627807 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469dbde-3b57-49c7-b344-a9b3ef1d54e7 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:28.663565 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 262/518] 10.222.0.45 () {56 vars in 1137 bytes} [Wed Jun 15 14:29:28 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:28.664378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469dbde-3b57-49c7-b344-a9b3ef1d54e7 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:28.680483 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feabae2-4bea-498b-b39d-8f0038104238 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:28.715332 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254195d5-142b-4355-bcf9-6936ab4c174d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:28.716119 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254195d5-142b-4355-bcf9-6936ab4c174d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:28.716397 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254195d5-142b-4355-bcf9-6936ab4c174d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:28.716672 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254195d5-142b-4355-bcf9-6936ab4c174d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:28.724099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254195d5-142b-4355-bcf9-6936ab4c174d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:28.725549 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254195d5-142b-4355-bcf9-6936ab4c174d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:28.739568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-254195d5-142b-4355-bcf9-6936ab4c174d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 14:29:28.739568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:28.739568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:28.739568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:28.739568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:28.739568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:28.739568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:28.739568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:28.739568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:28.739568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:28.739568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:28.739568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:28.739568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:29:28.739568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:29:28.739568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:29:28.739568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:29:28.740586 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:28.740586 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:28.740586 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:29:28.740586 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:28.740586 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:29:28.740586 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:28.740586 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:28.740586 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:28.740586 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:28.740586 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:28.740586 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:28.740586 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:28.740586 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:28.740586 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:28.740586 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:28.740586 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:28.740586 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:28.740586 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:28.741670 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:28.741670 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:28.741670 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:29:28.741670 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:29:28.741670 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:29:28.741670 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:29:28.741670 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:29:28.741670 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:29:28.741670 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 14:29:28.741670 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:28.741670 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 257/519] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Jun 15 14:29:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:28.741670 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254195d5-142b-4355-bcf9-6936ab4c174d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:28.756987 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:28.766345 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469dbde-3b57-49c7-b344-a9b3ef1d54e7 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:28.794555 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:28.802795 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36146fdc-58bb-4241-9e9a-a7aa776eb16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:28.803616 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36146fdc-58bb-4241-9e9a-a7aa776eb16c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:28.803899 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36146fdc-58bb-4241-9e9a-a7aa776eb16c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:28.804180 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36146fdc-58bb-4241-9e9a-a7aa776eb16c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:28.811386 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36146fdc-58bb-4241-9e9a-a7aa776eb16c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:28.812858 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36146fdc-58bb-4241-9e9a-a7aa776eb16c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:28.827335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-36146fdc-58bb-4241-9e9a-a7aa776eb16c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:28.827335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:28.827335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:28.827335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:28.827335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:28.827335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:28.827335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:28.827335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:28.827335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:28.827335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:28.827335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:28.827335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:28.827335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:29:28.827335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:29:28.827335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:29:28.828314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:29:28.828314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:28.828314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:28.828314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:28.828314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:28.828314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:28.828314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:28.828314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:28.828314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:28.828314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:28.828314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:28.828314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:28.828314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:28.828314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:28.828314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:28.828314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:28.828314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:28.833969 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:29:28.833969 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:29:28.833969 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:29:28.833969 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:29:28.833969 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:29:28.833969 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:29:28.833969 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:28.833969 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:28.833969 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 263/520] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:28.833969 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36146fdc-58bb-4241-9e9a-a7aa776eb16c None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:28.845161 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254195d5-142b-4355-bcf9-6936ab4c174d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:28.877977 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31251c-5b0c-4b8c-8d17-701f71c9ef17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:28.878937 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31251c-5b0c-4b8c-8d17-701f71c9ef17 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:28.879315 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31251c-5b0c-4b8c-8d17-701f71c9ef17 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:28.879660 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31251c-5b0c-4b8c-8d17-701f71c9ef17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:28.890567 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a31251c-5b0c-4b8c-8d17-701f71c9ef17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:28.892032 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a31251c-5b0c-4b8c-8d17-701f71c9ef17 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:28.908697 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-5a31251c-5b0c-4b8c-8d17-701f71c9ef17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:28.908697 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:28.908697 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:28.908697 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:28.908697 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:28.908697 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:28.908697 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:28.908697 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:28.908697 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:28.908697 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:28.908697 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:28.908697 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:28.908697 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:28.908697 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:28.908697 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:28.909682 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:28.909682 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:28.909682 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:28.909682 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:28.909682 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:28.909682 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:28.909682 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:28.909682 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:28.909682 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:28.909682 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:28.909682 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:28.909682 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:28.909682 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:28.909682 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:28.909682 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:28.909682 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:28.909682 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:28.910720 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:28.910720 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:28.910720 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:28.910720 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:28.910720 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 258/521] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:28.910720 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31251c-5b0c-4b8c-8d17-701f71c9ef17 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:28.930746 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36146fdc-58bb-4241-9e9a-a7aa776eb16c None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:28.965197 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b0488-ca19-469b-8c35-1cdfe64a993f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:28.965853 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b0488-ca19-469b-8c35-1cdfe64a993f None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:28.966139 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b0488-ca19-469b-8c35-1cdfe64a993f None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:28.966411 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b0488-ca19-469b-8c35-1cdfe64a993f None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:28.967302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b0488-ca19-469b-8c35-1cdfe64a993f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:28.968734 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b0488-ca19-469b-8c35-1cdfe64a993f None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:28.987782 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 264/522] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Jun 15 14:29:28 2022] POST /identity/v3/users => generated 247 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:29:28.988560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b0488-ca19-469b-8c35-1cdfe64a993f None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:29.000444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:29.026724 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31251c-5b0c-4b8c-8d17-701f71c9ef17 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:29.028872 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:29.070793 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df8a331-0aa2-42bb-9fbf-a27d424c7597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:29.071763 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df8a331-0aa2-42bb-9fbf-a27d424c7597 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:29.072184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df8a331-0aa2-42bb-9fbf-a27d424c7597 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:29.072328 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df8a331-0aa2-42bb-9fbf-a27d424c7597 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:29.107651 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b0488-ca19-469b-8c35-1cdfe64a993f None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:29.146359 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5a416-9b91-4b4e-a1bd-8a3a9ed081e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:29.147061 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5a416-9b91-4b4e-a1bd-8a3a9ed081e0 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:29.147351 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5a416-9b91-4b4e-a1bd-8a3a9ed081e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:29.147626 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5a416-9b91-4b4e-a1bd-8a3a9ed081e0 None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:29.148461 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b5a416-9b91-4b4e-a1bd-8a3a9ed081e0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:29.148773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b5a416-9b91-4b4e-a1bd-8a3a9ed081e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:29.150236 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b5a416-9b91-4b4e-a1bd-8a3a9ed081e0 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:29.154953 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df8a331-0aa2-42bb-9fbf-a27d424c7597 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:29.156436 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df8a331-0aa2-42bb-9fbf-a27d424c7597 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:29.170987 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 265/523] 10.222.0.45 () {56 vars in 1139 bytes} [Wed Jun 15 14:29:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:29.171491 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5a416-9b91-4b4e-a1bd-8a3a9ed081e0 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:29.247867 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:29.270163 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 259/524] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:29.271359 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df8a331-0aa2-42bb-9fbf-a27d424c7597 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:29.273662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:29.294575 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5a416-9b91-4b4e-a1bd-8a3a9ed081e0 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:29.333110 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f8a48-3614-4742-99df-863a971a17b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:29.334056 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f8a48-3614-4742-99df-863a971a17b6 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:29.334344 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f8a48-3614-4742-99df-863a971a17b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:29.334766 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f8a48-3614-4742-99df-863a971a17b6 None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:29.336118 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f8a48-3614-4742-99df-863a971a17b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:29.336446 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f8a48-3614-4742-99df-863a971a17b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:29.338356 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f8a48-3614-4742-99df-863a971a17b6 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:29.363935 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 266/525] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:29:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:29.363935 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f8a48-3614-4742-99df-863a971a17b6 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:29.378713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df8a331-0aa2-42bb-9fbf-a27d424c7597 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:29.415216 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a437603-d428-4a96-8938-33151317dfba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:29.415880 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a437603-d428-4a96-8938-33151317dfba None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:29.416158 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a437603-d428-4a96-8938-33151317dfba None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:29.416429 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a437603-d428-4a96-8938-33151317dfba None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:29.417132 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a437603-d428-4a96-8938-33151317dfba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:29.417967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a437603-d428-4a96-8938-33151317dfba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:29.419554 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a437603-d428-4a96-8938-33151317dfba None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:29.438851 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 260/526] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:29.439922 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a437603-d428-4a96-8938-33151317dfba None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:29.472261 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f8a48-3614-4742-99df-863a971a17b6 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:29.503587 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:29.506811 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:29.513177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68fcc7-fa26-458b-beed-8166da3f7852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:29.513831 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68fcc7-fa26-458b-beed-8166da3f7852 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:29.514121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68fcc7-fa26-458b-beed-8166da3f7852 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:29.514395 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68fcc7-fa26-458b-beed-8166da3f7852 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:29.553019 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a437603-d428-4a96-8938-33151317dfba None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:29.588526 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68fcc7-fa26-458b-beed-8166da3f7852 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:29.590088 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c92b0f-e2a9-4eb1-a9b8-21a5da3fbda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:29.590494 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68fcc7-fa26-458b-beed-8166da3f7852 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:29.591310 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c92b0f-e2a9-4eb1-a9b8-21a5da3fbda8 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:29.591762 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c92b0f-e2a9-4eb1-a9b8-21a5da3fbda8 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:29.592323 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c92b0f-e2a9-4eb1-a9b8-21a5da3fbda8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:29.603727 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c92b0f-e2a9-4eb1-a9b8-21a5da3fbda8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:29.606183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c92b0f-e2a9-4eb1-a9b8-21a5da3fbda8 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:29.624123 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-21c92b0f-e2a9-4eb1-a9b8-21a5da3fbda8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:29.624123 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:29.624123 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:29.624123 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:29.624123 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:29.624123 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:29.624123 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:29.624123 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:29.624123 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:29.624123 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:29.624123 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:29.624123 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:29.624123 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:29.624123 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:29.624123 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:29.624942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:29.624942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:29.624942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:29.624942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:29.624942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:29.624942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:29.624942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:29.624942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:29.624942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:29.624942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:29.624942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:29.624942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:29.624942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:29.624942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:29.624942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:29.624942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:29.624942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:29.625848 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:29.625848 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:29.625848 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:29.625848 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:29.627688 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 261/527] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:29.628592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c92b0f-e2a9-4eb1-a9b8-21a5da3fbda8 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:29.700585 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 267/528] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:29.701433 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68fcc7-fa26-458b-beed-8166da3f7852 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:29.743115 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:29.746586 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c92b0f-e2a9-4eb1-a9b8-21a5da3fbda8 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:29.760697 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:29.784902 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c1b6c-f346-43f1-8891-762cf81eacad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:29.785564 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c1b6c-f346-43f1-8891-762cf81eacad None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:29.785856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c1b6c-f346-43f1-8891-762cf81eacad None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:29.786237 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c1b6c-f346-43f1-8891-762cf81eacad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:29.786974 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c1b6c-f346-43f1-8891-762cf81eacad None admin] RBAC: Adding query filter params (role.id=13dcf68cca9c44888c0fd27a78db3dbe, scope.project.id=d488c79cb1704e9496fc196d8aa6c104, user.id=f3185bd2458c4e22830975700888e9be) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:29.787320 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c1b6c-f346-43f1-8891-762cf81eacad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:29.788767 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c1b6c-f346-43f1-8891-762cf81eacad None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:29.803028 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 262/529] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:29:29 2022] GET /identity/v3/role_assignments?user.id=f3185bd2458c4e22830975700888e9be&scope.project.id=d488c79cb1704e9496fc196d8aa6c104&role.id=13dcf68cca9c44888c0fd27a78db3dbe => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:29.803028 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c1b6c-f346-43f1-8891-762cf81eacad None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:29.832909 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68fcc7-fa26-458b-beed-8166da3f7852 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:29.878136 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c88b0e7-cb7c-4ed6-8fba-728b7095c7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:29.879349 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88b0e7-cb7c-4ed6-8fba-728b7095c7de None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:29.879737 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88b0e7-cb7c-4ed6-8fba-728b7095c7de None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:29.880087 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88b0e7-cb7c-4ed6-8fba-728b7095c7de None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:29.890092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c88b0e7-cb7c-4ed6-8fba-728b7095c7de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:29.892518 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c88b0e7-cb7c-4ed6-8fba-728b7095c7de None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:29.912843 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-3c88b0e7-cb7c-4ed6-8fba-728b7095c7de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:29.912843 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:29.912843 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:29.912843 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:29.912843 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:29.912843 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:29.912843 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:29.912843 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:29.912843 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:29.912843 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:29.912843 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:29.912843 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:29.912843 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:29:29.912843 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:29:29.912843 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:29:29.913923 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:29:29.913923 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:29.913923 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:29.913923 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:29.913923 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:29.913923 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:29.913923 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:29.913923 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:29.913923 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:29.913923 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:29.913923 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:29.913923 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:29.913923 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:29.913923 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:29.913923 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:29.913923 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:29.913923 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:29.914908 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:29:29.914908 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:29:29.914908 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:29:29.914908 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:29:29.914908 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:29:29.914908 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:29:29.914908 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:29.914908 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:29.917642 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 268/530] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:29.917925 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c88b0e7-cb7c-4ed6-8fba-728b7095c7de None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:29.918941 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c1b6c-f346-43f1-8891-762cf81eacad None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:29.959356 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e187f5-dbe4-4a77-b93d-8250076e60af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:29.960139 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e187f5-dbe4-4a77-b93d-8250076e60af None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:29.960500 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e187f5-dbe4-4a77-b93d-8250076e60af None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:29.960856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e187f5-dbe4-4a77-b93d-8250076e60af None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:29.961683 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e187f5-dbe4-4a77-b93d-8250076e60af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:29.962072 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e187f5-dbe4-4a77-b93d-8250076e60af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:29.963709 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e187f5-dbe4-4a77-b93d-8250076e60af None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:29.981206 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 263/531] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:29.982093 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e187f5-dbe4-4a77-b93d-8250076e60af None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:30.018408 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:30.033759 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:30.045913 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c88b0e7-cb7c-4ed6-8fba-728b7095c7de None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:30.094310 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8398d429-bd10-49ab-8632-f0ec5500a7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:30.095769 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398d429-bd10-49ab-8632-f0ec5500a7cd None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:30.096334 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398d429-bd10-49ab-8632-f0ec5500a7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:30.096843 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398d429-bd10-49ab-8632-f0ec5500a7cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:30.101517 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e187f5-dbe4-4a77-b93d-8250076e60af None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:30.111888 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8398d429-bd10-49ab-8632-f0ec5500a7cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:30.114232 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8398d429-bd10-49ab-8632-f0ec5500a7cd None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:30.140605 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-8398d429-bd10-49ab-8632-f0ec5500a7cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:30.140605 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:30.140605 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:30.140605 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:30.140605 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:30.140605 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:30.140605 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:30.140605 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:30.140605 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:30.140605 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:30.140605 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:30.140605 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:30.140605 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:30.140605 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:30.140605 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:30.141641 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:30.141641 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:30.141641 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:30.141641 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:30.141641 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:30.141641 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:30.141641 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:30.141641 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:30.141641 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:30.141641 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:30.141641 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:30.141641 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:30.141641 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:30.141641 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:30.141641 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:30.141641 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:30.141641 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:30.142643 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:30.142643 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:30.142643 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:30.142643 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:30.143219 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d71027-d97b-489f-aab9-a4a61289017a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:30.144248 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d71027-d97b-489f-aab9-a4a61289017a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:30.144628 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d71027-d97b-489f-aab9-a4a61289017a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:30.145007 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d71027-d97b-489f-aab9-a4a61289017a None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:30.149432 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d71027-d97b-489f-aab9-a4a61289017a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:30.149432 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d71027-d97b-489f-aab9-a4a61289017a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:30.149432 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d71027-d97b-489f-aab9-a4a61289017a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:30.150234 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 269/532] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:30.151296 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8398d429-bd10-49ab-8632-f0ec5500a7cd None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:30.172481 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 264/533] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:29:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:30.173864 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d71027-d97b-489f-aab9-a4a61289017a None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:30.284761 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8398d429-bd10-49ab-8632-f0ec5500a7cd None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:30.290411 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:30.295297 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d71027-d97b-489f-aab9-a4a61289017a None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:30.331232 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ba6db-ee40-48d0-889b-0ce8d6db4a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:30.332213 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f0a793-9d11-4a9a-8227-d69f1bb09d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:30.332586 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ba6db-ee40-48d0-889b-0ce8d6db4a10 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:30.333011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0a793-9d11-4a9a-8227-d69f1bb09d21 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:30.333133 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ba6db-ee40-48d0-889b-0ce8d6db4a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:30.333289 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0a793-9d11-4a9a-8227-d69f1bb09d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:30.333562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0a793-9d11-4a9a-8227-d69f1bb09d21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:30.333849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ba6db-ee40-48d0-889b-0ce8d6db4a10 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:30.334891 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ba6db-ee40-48d0-889b-0ce8d6db4a10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:30.335351 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ba6db-ee40-48d0-889b-0ce8d6db4a10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:30.336953 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ba6db-ee40-48d0-889b-0ce8d6db4a10 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:30.342095 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f0a793-9d11-4a9a-8227-d69f1bb09d21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:30.343582 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f0a793-9d11-4a9a-8227-d69f1bb09d21 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:30.345891 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:30.356352 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 265/534] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:30.357427 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ba6db-ee40-48d0-889b-0ce8d6db4a10 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:30.362727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-e0f0a793-9d11-4a9a-8227-d69f1bb09d21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:30.362727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:30.362727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:30.362727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:30.362727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:30.362727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:30.362727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:30.362727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:30.362727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:30.362727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:30.362727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:30.362727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:30.362727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:29:30.362727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:29:30.362727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:29:30.363720 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:29:30.363720 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:30.363720 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:30.363720 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:30.363720 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:30.363720 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:30.363720 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:30.363720 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:30.363720 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:30.363720 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:30.363720 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:30.363720 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:30.363720 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:30.363720 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:30.363720 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:30.363720 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:30.363720 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:30.364667 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:29:30.364667 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:29:30.364667 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:29:30.364667 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:29:30.364667 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:29:30.364667 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:29:30.364667 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:30.364667 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:30.364667 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 270/535] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:30.365105 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f0a793-9d11-4a9a-8227-d69f1bb09d21 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:30.479395 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f0a793-9d11-4a9a-8227-d69f1bb09d21 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:30.490494 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ba6db-ee40-48d0-889b-0ce8d6db4a10 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:30.525822 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd320a-b57b-41d0-a981-ce2c10c1b334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:30.526622 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd320a-b57b-41d0-a981-ce2c10c1b334 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:30.527450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd320a-b57b-41d0-a981-ce2c10c1b334 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:30.527726 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd320a-b57b-41d0-a981-ce2c10c1b334 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:30.527880 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8d9ab-b19a-473e-ae77-839688ff5365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:30.528835 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8d9ab-b19a-473e-ae77-839688ff5365 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:30.529264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8d9ab-b19a-473e-ae77-839688ff5365 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:30.529667 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8d9ab-b19a-473e-ae77-839688ff5365 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:30.530784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac8d9ab-b19a-473e-ae77-839688ff5365 None admin] RBAC: Adding query filter params (role.id=13dcf68cca9c44888c0fd27a78db3dbe, scope.project.id=d488c79cb1704e9496fc196d8aa6c104, user.id=e4f10214b2bb4eafb3dff7adf71b9bf9) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:30.531253 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac8d9ab-b19a-473e-ae77-839688ff5365 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:30.532861 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac8d9ab-b19a-473e-ae77-839688ff5365 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:30.535035 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd320a-b57b-41d0-a981-ce2c10c1b334 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:30.537035 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd320a-b57b-41d0-a981-ce2c10c1b334 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:30.539571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 266/536] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:29:30 2022] GET /identity/v3/role_assignments?user.id=e4f10214b2bb4eafb3dff7adf71b9bf9&scope.project.id=d488c79cb1704e9496fc196d8aa6c104&role.id=13dcf68cca9c44888c0fd27a78db3dbe => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:30.540638 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8d9ab-b19a-473e-ae77-839688ff5365 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:30.555065 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-5acd320a-b57b-41d0-a981-ce2c10c1b334 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:30.555065 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:30.555065 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:30.555065 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:30.555065 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:30.555065 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:30.555065 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:30.555065 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:30.555065 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:30.555065 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:30.555065 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:30.555065 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:30.555065 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:29:30.555065 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:29:30.555065 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:29:30.556046 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:29:30.556046 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:30.556046 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:30.556046 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:30.556046 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:30.556046 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:30.556046 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:30.556046 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:30.556046 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:30.556046 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:30.556046 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:30.556046 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:30.556046 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:30.556046 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:30.556046 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:30.556046 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:30.556046 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:30.556857 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:29:30.556857 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:29:30.556857 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:29:30.556857 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:29:30.556857 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:29:30.556857 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:29:30.556857 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:30.556857 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:30.556857 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 271/537] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:30.564104 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd320a-b57b-41d0-a981-ce2c10c1b334 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:30.591265 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:30.682357 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:30.741123 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8d9ab-b19a-473e-ae77-839688ff5365 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:30.768591 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd320a-b57b-41d0-a981-ce2c10c1b334 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:30.789281 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa5336-da99-4e8b-a28a-7361ed03675a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:30.789978 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa5336-da99-4e8b-a28a-7361ed03675a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:30.790501 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa5336-da99-4e8b-a28a-7361ed03675a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:30.790791 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa5336-da99-4e8b-a28a-7361ed03675a None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:30.792030 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffa5336-da99-4e8b-a28a-7361ed03675a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:30.792371 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffa5336-da99-4e8b-a28a-7361ed03675a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:30.794335 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffa5336-da99-4e8b-a28a-7361ed03675a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:30.812484 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 267/538] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:29:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:30.823656 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa5336-da99-4e8b-a28a-7361ed03675a None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:30.829208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939b255-9eb6-4bd4-bf5d-927060070fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:30.830152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939b255-9eb6-4bd4-bf5d-927060070fee None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:30.830617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939b255-9eb6-4bd4-bf5d-927060070fee None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:30.831092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939b255-9eb6-4bd4-bf5d-927060070fee None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:30.832059 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4939b255-9eb6-4bd4-bf5d-927060070fee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:30.832536 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4939b255-9eb6-4bd4-bf5d-927060070fee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:30.834165 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4939b255-9eb6-4bd4-bf5d-927060070fee None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:30.860665 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 272/539] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:29:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:30.866498 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939b255-9eb6-4bd4-bf5d-927060070fee None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:30.921764 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:30.948972 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa5336-da99-4e8b-a28a-7361ed03675a None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:31.009907 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939b255-9eb6-4bd4-bf5d-927060070fee None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:31.016864 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd2f98-8d05-476a-9358-102d2885bf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:31.017765 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd2f98-8d05-476a-9358-102d2885bf28 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:31.018224 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd2f98-8d05-476a-9358-102d2885bf28 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:31.018633 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd2f98-8d05-476a-9358-102d2885bf28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:31.019923 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd2f98-8d05-476a-9358-102d2885bf28 None admin] RBAC: Adding query filter params (role.id=13dcf68cca9c44888c0fd27a78db3dbe, scope.project.id=d488c79cb1704e9496fc196d8aa6c104, user.id=b47cec76cb1c4afb8e011ff04a885f96) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:31.020395 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd2f98-8d05-476a-9358-102d2885bf28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:31.021980 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd2f98-8d05-476a-9358-102d2885bf28 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:31.030016 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:31.034585 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 268/540] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:29:30 2022] GET /identity/v3/role_assignments?user.id=b47cec76cb1c4afb8e011ff04a885f96&scope.project.id=d488c79cb1704e9496fc196d8aa6c104&role.id=13dcf68cca9c44888c0fd27a78db3dbe => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:31.050506 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b9848-f24e-4d3b-b328-6dcb79664798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:31.051562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b9848-f24e-4d3b-b328-6dcb79664798 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:31.052014 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b9848-f24e-4d3b-b328-6dcb79664798 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:31.052470 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b9848-f24e-4d3b-b328-6dcb79664798 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:31.053472 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b9848-f24e-4d3b-b328-6dcb79664798 None admin] RBAC: Adding query filter params (role.id=13dcf68cca9c44888c0fd27a78db3dbe, scope.project.id=d488c79cb1704e9496fc196d8aa6c104, user.id=975ae2790ac849ff88e991231671f510) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:31.053973 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b9848-f24e-4d3b-b328-6dcb79664798 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:31.055617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b9848-f24e-4d3b-b328-6dcb79664798 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:31.062396 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 273/541] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:29:30 2022] GET /identity/v3/role_assignments?user.id=975ae2790ac849ff88e991231671f510&scope.project.id=d488c79cb1704e9496fc196d8aa6c104&role.id=13dcf68cca9c44888c0fd27a78db3dbe => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:31.229424 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d6045-bf33-45cc-9ec8-827bc5a3ba6a None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:31.229718 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d6045-bf33-45cc-9ec8-827bc5a3ba6a None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:31.230575 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d6045-bf33-45cc-9ec8-827bc5a3ba6a None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:31.232354 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 269/542] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:31.238350 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52b6db-92e2-42e8-bd88-2c97c885953a None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:31.238957 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52b6db-92e2-42e8-bd88-2c97c885953a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:31.239478 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52b6db-92e2-42e8-bd88-2c97c885953a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:31.257426 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-8c52b6db-92e2-42e8-bd88-2c97c885953a None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:31.259122 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-8c52b6db-92e2-42e8-bd88-2c97c885953a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:31.336291 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 274/543] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:31.362098 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734d6045-bf33-45cc-9ec8-827bc5a3ba6a None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:31.470538 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734d6045-bf33-45cc-9ec8-827bc5a3ba6a None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:31.506248 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d264d849-0dc3-4c3c-819b-40be82816a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:31.507083 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d264d849-0dc3-4c3c-819b-40be82816a02 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:31.507373 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d264d849-0dc3-4c3c-819b-40be82816a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:31.507703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d264d849-0dc3-4c3c-819b-40be82816a02 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:31.519282 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d264d849-0dc3-4c3c-819b-40be82816a02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:31.527905 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-d264d849-0dc3-4c3c-819b-40be82816a02 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:31.527905 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:31.527905 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:31.527905 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:31.527905 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:31.527905 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:31.527905 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:31.527905 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:31.527905 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:31.527905 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:31.527905 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:31.527905 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:31.527905 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:31.527905 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:31.527905 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:31.527905 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:31.528756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:31.528756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:31.528756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:31.528756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:31.528756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:31.528756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:31.528756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:31.528756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:31.528756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:31.528756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:31.528756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:31.528756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:31.528756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:31.528756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:31.528756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:31.528756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:31.529962 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:31.529962 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:31.529962 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:31.529962 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:31.529962 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:31.529962 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:31.529962 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:31.529962 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:31.529962 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:31.529962 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:31.529962 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:31.529962 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:31.529962 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 270/544] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Jun 15 14:29:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:31.535567 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52b6db-92e2-42e8-bd88-2c97c885953a None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:31.678079 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52b6db-92e2-42e8-bd88-2c97c885953a None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:31.697157 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:31.736747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c648ecf-7f7e-433c-a5df-8db72975baa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:31.737494 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c648ecf-7f7e-433c-a5df-8db72975baa8 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:31.737782 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c648ecf-7f7e-433c-a5df-8db72975baa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:31.738061 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c648ecf-7f7e-433c-a5df-8db72975baa8 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:31.738875 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c648ecf-7f7e-433c-a5df-8db72975baa8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:31.739210 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c648ecf-7f7e-433c-a5df-8db72975baa8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:31.740609 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c648ecf-7f7e-433c-a5df-8db72975baa8 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:31.759141 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 275/545] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Jun 15 14:29:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:31.760109 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d264d849-0dc3-4c3c-819b-40be82816a02 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:31.774160 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:31.880435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d264d849-0dc3-4c3c-819b-40be82816a02 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:31.914486 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd39414-e1d8-4c3f-98ab-e9289e678fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:31.915303 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd39414-e1d8-4c3f-98ab-e9289e678fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:31.915581 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd39414-e1d8-4c3f-98ab-e9289e678fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:31.915847 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd39414-e1d8-4c3f-98ab-e9289e678fc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:31.932225 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd39414-e1d8-4c3f-98ab-e9289e678fc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:31.933657 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd39414-e1d8-4c3f-98ab-e9289e678fc4 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:31.953466 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-9bd39414-e1d8-4c3f-98ab-e9289e678fc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:31.953466 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:31.953466 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:31.953466 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:31.953466 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:31.953466 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:31.953466 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:31.953466 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:31.953466 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:31.953466 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:31.953466 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:31.953466 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:31.953466 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:31.953466 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:31.953466 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:31.954366 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:31.954366 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:31.954366 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:31.954366 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:31.954366 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:31.954366 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:31.954366 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:31.954366 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:31.954366 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:31.954366 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:31.954366 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:31.954366 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:31.954366 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:31.954366 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:31.954366 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:31.954366 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:31.954366 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:31.955224 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:31.955224 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:31.955224 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:31.955224 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:31.955224 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 271/546] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:31.960521 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c648ecf-7f7e-433c-a5df-8db72975baa8 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:32.116718 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c648ecf-7f7e-433c-a5df-8db72975baa8 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:32.169722 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0c0349-5a23-42fc-9e94-2c6ae44d082a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:32.171078 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c0349-5a23-42fc-9e94-2c6ae44d082a None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:32.171544 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c0349-5a23-42fc-9e94-2c6ae44d082a None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:32.171992 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c0349-5a23-42fc-9e94-2c6ae44d082a None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:32.172961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0c0349-5a23-42fc-9e94-2c6ae44d082a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:32.173454 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0c0349-5a23-42fc-9e94-2c6ae44d082a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:32.175739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0c0349-5a23-42fc-9e94-2c6ae44d082a None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:32.189355 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 276/547] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:32.194415 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd39414-e1d8-4c3f-98ab-e9289e678fc4 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:32.281109 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:32.338138 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd39414-e1d8-4c3f-98ab-e9289e678fc4 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:32.379984 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf3977b-62b9-471d-a722-c20c5b961c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:32.381125 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3977b-62b9-471d-a722-c20c5b961c8f None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:32.381592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3977b-62b9-471d-a722-c20c5b961c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:32.382090 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3977b-62b9-471d-a722-c20c5b961c8f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:32.394706 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf3977b-62b9-471d-a722-c20c5b961c8f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:32.396426 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf3977b-62b9-471d-a722-c20c5b961c8f None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:32.411643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-0cf3977b-62b9-471d-a722-c20c5b961c8f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:29:32.411643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:32.411643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:32.411643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:32.411643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:32.411643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:32.411643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:32.411643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:32.411643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:32.411643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:32.411643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:32.411643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:32.411643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:29:32.411643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:29:32.411643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:29:32.411643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:29:32.412498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:32.412498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:32.412498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:29:32.412498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:32.412498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:29:32.412498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:32.412498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:32.412498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:32.412498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:32.412498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:32.412498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:32.412498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:32.412498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:32.412498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:32.412498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:32.412498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:32.412498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:32.412498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:32.413876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:32.413876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:32.413876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:29:32.413876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:29:32.413876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:29:32.413876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:29:32.413876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:29:32.413876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:29:32.413876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:29:32.413876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:32.416292 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 272/548] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:29:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:32.424302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0c0349-5a23-42fc-9e94-2c6ae44d082a None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:32.442662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1ae97-7e7f-409b-9be1-071276067e7a None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:32.442981 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1ae97-7e7f-409b-9be1-071276067e7a None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:32.443290 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1ae97-7e7f-409b-9be1-071276067e7a None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:32.444802 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 273/549] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:32.451014 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611f17a-8dc4-4a5a-88f9-77d68c36fc56 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:32.451304 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611f17a-8dc4-4a5a-88f9-77d68c36fc56 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:32.451582 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611f17a-8dc4-4a5a-88f9-77d68c36fc56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:32.481314 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-e611f17a-8dc4-4a5a-88f9-77d68c36fc56 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:32.482533 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e611f17a-8dc4-4a5a-88f9-77d68c36fc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:32.503475 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:32.611329 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0c0349-5a23-42fc-9e94-2c6ae44d082a None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:32.616656 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 274/550] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:32.647928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece54cee-66ac-487e-af97-fccec23a5b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:32.648613 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece54cee-66ac-487e-af97-fccec23a5b77 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:32.648900 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece54cee-66ac-487e-af97-fccec23a5b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:32.649183 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece54cee-66ac-487e-af97-fccec23a5b77 None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:32.650027 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece54cee-66ac-487e-af97-fccec23a5b77 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:32.650343 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece54cee-66ac-487e-af97-fccec23a5b77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:32.652342 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece54cee-66ac-487e-af97-fccec23a5b77 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:32.660497 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e611f17a-8dc4-4a5a-88f9-77d68c36fc56 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:32.683349 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 277/551] 10.222.0.45 () {56 vars in 1135 bytes} [Wed Jun 15 14:29:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:32.688816 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece54cee-66ac-487e-af97-fccec23a5b77 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:32.758812 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:32.818208 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e611f17a-8dc4-4a5a-88f9-77d68c36fc56 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:32.827064 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:32.842249 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece54cee-66ac-487e-af97-fccec23a5b77 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:32.867968 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4b76e-7b9a-4951-b822-21e2a7559792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:32.868771 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4b76e-7b9a-4951-b822-21e2a7559792 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:32.869053 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4b76e-7b9a-4951-b822-21e2a7559792 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:32.869328 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4b76e-7b9a-4951-b822-21e2a7559792 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:32.880845 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de4b76e-7b9a-4951-b822-21e2a7559792 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:32.889145 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-0de4b76e-7b9a-4951-b822-21e2a7559792 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:32.889145 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:32.889145 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:32.889145 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:32.889145 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:32.889145 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:32.889145 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:32.889145 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:32.889145 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:32.889145 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:32.889145 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:32.889145 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:32.889145 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:32.889145 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:32.889145 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:32.889145 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:32.890038 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:32.890038 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:32.890038 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:32.890038 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:32.890038 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:32.890038 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:32.890038 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:32.890038 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:32.890038 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:32.890038 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:32.890038 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:32.890038 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:32.890038 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:32.890038 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:32.890038 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:32.890038 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:32.890931 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:32.890931 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:32.890931 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:32.890931 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:32.890931 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:32.890931 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:32.890931 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:32.890931 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:32.890931 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:32.890931 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:32.890931 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:32.890931 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:32.890931 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 275/552] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Jun 15 14:29:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:29:32.900099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4b76e-7b9a-4951-b822-21e2a7559792 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:32.947568 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26106a35-eb74-4a66-ae50-a27c3d0692d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:32.948224 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26106a35-eb74-4a66-ae50-a27c3d0692d7 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:32.948563 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26106a35-eb74-4a66-ae50-a27c3d0692d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:32.948870 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26106a35-eb74-4a66-ae50-a27c3d0692d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:33.074009 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4b76e-7b9a-4951-b822-21e2a7559792 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:33.088117 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26106a35-eb74-4a66-ae50-a27c3d0692d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:33.090163 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26106a35-eb74-4a66-ae50-a27c3d0692d7 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:33.094042 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:33.149683 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4536f4-09e2-4dd0-bc7c-2322da60f155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:33.152276 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4536f4-09e2-4dd0-bc7c-2322da60f155 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:33.152765 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4536f4-09e2-4dd0-bc7c-2322da60f155 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:33.153255 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4536f4-09e2-4dd0-bc7c-2322da60f155 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:33.155218 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4536f4-09e2-4dd0-bc7c-2322da60f155 None admin] RBAC: Adding query filter params (name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:33.157072 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4536f4-09e2-4dd0-bc7c-2322da60f155 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:33.158677 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4536f4-09e2-4dd0-bc7c-2322da60f155 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:33.176568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 276/553] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Jun 15 14:29:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:33.182249 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4536f4-09e2-4dd0-bc7c-2322da60f155 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:33.225321 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:33.236764 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 278/554] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 14:29:33.238367 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add01abe-2351-4f9c-ba63-b25fced07684 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:33.238844 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add01abe-2351-4f9c-ba63-b25fced07684 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:33.239290 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add01abe-2351-4f9c-ba63-b25fced07684 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:33.240959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 279/555] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:33.241802 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add01abe-2351-4f9c-ba63-b25fced07684 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:33.309771 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4536f4-09e2-4dd0-bc7c-2322da60f155 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:33.351192 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea2bee-3dbe-4314-8d87-4880db050245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:33.352348 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea2bee-3dbe-4314-8d87-4880db050245 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:33.352768 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea2bee-3dbe-4314-8d87-4880db050245 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:33.353179 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea2bee-3dbe-4314-8d87-4880db050245 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:33.366094 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea2bee-3dbe-4314-8d87-4880db050245 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:33.367933 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea2bee-3dbe-4314-8d87-4880db050245 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:33.383011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add01abe-2351-4f9c-ba63-b25fced07684 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:33.390567 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-8aea2bee-3dbe-4314-8d87-4880db050245 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:33.390567 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:33.390567 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:33.390567 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:33.390567 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:33.390567 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:33.390567 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:33.390567 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:33.390567 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:33.390567 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:33.390567 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:33.390567 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:33.390567 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:33.390567 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:33.390567 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:33.391443 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:33.391443 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:33.391443 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:33.391443 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:33.391443 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:33.391443 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:33.391443 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:33.391443 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:33.391443 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:33.391443 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:33.391443 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:33.391443 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:33.391443 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:33.391443 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:33.391443 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:33.391443 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:33.391443 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:33.392277 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:33.392277 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:33.392277 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:33.392277 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:33.392277 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 277/556] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:33.394673 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3300f-711b-45bf-9873-51c7d5abcadb None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:33.395048 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3300f-711b-45bf-9873-51c7d5abcadb None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:33.395404 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3300f-711b-45bf-9873-51c7d5abcadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:33.418842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-f6a3300f-711b-45bf-9873-51c7d5abcadb None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:33.420816 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f6a3300f-711b-45bf-9873-51c7d5abcadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:33.425654 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7d0ff-953c-4a93-aad6-01abdf80fc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:33.426709 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7d0ff-953c-4a93-aad6-01abdf80fc41 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:33.427228 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7d0ff-953c-4a93-aad6-01abdf80fc41 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:33.427696 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7d0ff-953c-4a93-aad6-01abdf80fc41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:33.440742 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c7d0ff-953c-4a93-aad6-01abdf80fc41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:33.442678 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c7d0ff-953c-4a93-aad6-01abdf80fc41 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:33.443218 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:33.464830 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-52c7d0ff-953c-4a93-aad6-01abdf80fc41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:33.464830 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:33.464830 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:33.464830 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:33.464830 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:33.464830 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:33.464830 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:33.464830 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:33.464830 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:33.464830 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:33.464830 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:33.464830 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:33.464830 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:33.464830 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:33.464830 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:33.466280 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:33.466280 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:33.466280 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:33.466280 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:33.466280 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:33.466280 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:33.466280 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:33.466280 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:33.466280 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:33.466280 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:33.466280 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:33.466280 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:33.466280 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:33.466280 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:33.466280 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:33.466280 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:33.466280 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:33.467615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:33.467615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:33.467615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:33.467615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:33.469682 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 280/557] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:33.470804 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7d0ff-953c-4a93-aad6-01abdf80fc41 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:33.507032 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 278/558] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:33.508063 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a3300f-711b-45bf-9873-51c7d5abcadb None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:33.521162 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:33.595014 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7d0ff-953c-4a93-aad6-01abdf80fc41 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:33.627844 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a3300f-711b-45bf-9873-51c7d5abcadb None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:33.641029 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca35459-9dd1-4388-8555-7f501241e9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:33.642574 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca35459-9dd1-4388-8555-7f501241e9fd None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:33.644283 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca35459-9dd1-4388-8555-7f501241e9fd None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:33.644579 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca35459-9dd1-4388-8555-7f501241e9fd None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:33.645331 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca35459-9dd1-4388-8555-7f501241e9fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:33.645649 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca35459-9dd1-4388-8555-7f501241e9fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:33.647636 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca35459-9dd1-4388-8555-7f501241e9fd None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:33.660409 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 281/559] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:33.661231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca35459-9dd1-4388-8555-7f501241e9fd None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:33.671107 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db17c848-5c98-4cf3-9972-139fe11c3f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:33.671772 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17c848-5c98-4cf3-9972-139fe11c3f21 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:33.672054 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17c848-5c98-4cf3-9972-139fe11c3f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:33.672334 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17c848-5c98-4cf3-9972-139fe11c3f21 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:33.673056 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db17c848-5c98-4cf3-9972-139fe11c3f21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:33.673361 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db17c848-5c98-4cf3-9972-139fe11c3f21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:33.674735 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db17c848-5c98-4cf3-9972-139fe11c3f21 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:33.691145 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 279/560] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:33.691145 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0440a-4dd2-4a31-8feb-58ecb8fa3745 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:33.691145 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0440a-4dd2-4a31-8feb-58ecb8fa3745 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:33.691145 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0440a-4dd2-4a31-8feb-58ecb8fa3745 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:33.691145 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 280/561] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:33.691145 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af0440a-4dd2-4a31-8feb-58ecb8fa3745 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:33.710104 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:33.789818 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca35459-9dd1-4388-8555-7f501241e9fd None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:33.793525 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:33.819107 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af0440a-4dd2-4a31-8feb-58ecb8fa3745 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:33.835542 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0691d75-4217-456b-8c7b-2106fc4541c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:33.836345 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0691d75-4217-456b-8c7b-2106fc4541c7 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:33.836627 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0691d75-4217-456b-8c7b-2106fc4541c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:33.836898 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0691d75-4217-456b-8c7b-2106fc4541c7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:33.848111 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0691d75-4217-456b-8c7b-2106fc4541c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:33.855919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-a0691d75-4217-456b-8c7b-2106fc4541c7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:33.855919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:33.855919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:33.855919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:33.855919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:33.855919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:33.855919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:33.855919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:33.855919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:33.855919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:33.855919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:33.855919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:33.855919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:33.855919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:33.855919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:33.855919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:33.856860 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:33.856860 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:33.856860 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:33.856860 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:33.856860 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:33.856860 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:33.856860 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:33.856860 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:33.856860 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:33.856860 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:33.856860 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:33.856860 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:33.856860 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:33.856860 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:33.856860 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:33.856860 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:33.857678 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:33.857678 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:33.857678 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:33.857678 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:33.857678 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:33.857678 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:33.857678 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:33.857678 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:33.857678 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:33.857678 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:33.857678 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:33.857678 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:33.857678 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 282/562] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Jun 15 14:29:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:33.858361 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0691d75-4217-456b-8c7b-2106fc4541c7 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:33.863691 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a6bbf-8d42-4edd-8c21-f4e6414c1644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:33.864185 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a6bbf-8d42-4edd-8c21-f4e6414c1644 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:33.864471 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a6bbf-8d42-4edd-8c21-f4e6414c1644 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:33.864748 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a6bbf-8d42-4edd-8c21-f4e6414c1644 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:33.872952 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a6bbf-8d42-4edd-8c21-f4e6414c1644 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:33.874396 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a6bbf-8d42-4edd-8c21-f4e6414c1644 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:33.888868 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-006a6bbf-8d42-4edd-8c21-f4e6414c1644 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:29:33.888868 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:33.888868 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:33.888868 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:33.888868 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:33.888868 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:33.888868 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:33.888868 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:33.888868 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:33.888868 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:33.888868 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:33.888868 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:33.888868 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:29:33.888868 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:29:33.888868 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:29:33.888868 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:29:33.889856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:33.889856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:33.889856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:29:33.889856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:33.889856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:29:33.889856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:33.889856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:33.889856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:33.889856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:33.889856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:33.889856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:33.889856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:33.889856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:33.889856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:33.889856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:33.889856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:33.889856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:33.889856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:33.890765 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:33.890765 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:33.890765 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:29:33.890765 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:29:33.890765 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:29:33.890765 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:29:33.890765 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:29:33.890765 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:29:33.890765 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:29:33.890765 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:33.890765 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 281/563] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Jun 15 14:29:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:33.891385 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79bf69-0f40-4540-96da-219365b9194a None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:33.891453 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79bf69-0f40-4540-96da-219365b9194a None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:33.892049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79bf69-0f40-4540-96da-219365b9194a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:33.908286 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-9e79bf69-0f40-4540-96da-219365b9194a None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:33.909458 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-9e79bf69-0f40-4540-96da-219365b9194a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:33.958592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0691d75-4217-456b-8c7b-2106fc4541c7 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:33.975077 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 282/564] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:33.979135 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79bf69-0f40-4540-96da-219365b9194a None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:33.981483 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:33.992741 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd3570-f991-4618-bc49-2c5be032c2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:33.993587 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd3570-f991-4618-bc49-2c5be032c2be None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:33.993868 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd3570-f991-4618-bc49-2c5be032c2be None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:33.994167 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd3570-f991-4618-bc49-2c5be032c2be None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:34.000932 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd3570-f991-4618-bc49-2c5be032c2be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:34.002370 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd3570-f991-4618-bc49-2c5be032c2be None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:34.016532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-8fbd3570-f991-4618-bc49-2c5be032c2be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:34.016532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:34.016532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:34.016532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:34.016532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:34.016532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:34.016532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:34.016532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:34.016532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:34.016532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:34.016532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:34.016532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:34.016532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:29:34.016532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:29:34.016532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:29:34.017421 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:29:34.017421 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:34.017421 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:34.017421 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:34.017421 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:34.017421 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:34.017421 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:34.017421 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:34.017421 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:34.017421 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:34.017421 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:34.017421 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:34.017421 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:34.017421 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:34.017421 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:34.017421 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:34.017421 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:34.018465 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:29:34.018465 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:29:34.018465 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:29:34.018465 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:29:34.018465 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:29:34.018465 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:29:34.018465 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:34.018465 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:34.018465 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 283/565] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:34.018941 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83864000-e5a1-492f-bfb5-337c9ffc9dfc None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:34.019742 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83864000-e5a1-492f-bfb5-337c9ffc9dfc None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:34.019742 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83864000-e5a1-492f-bfb5-337c9ffc9dfc None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:34.023050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 284/566] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:34.023050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83864000-e5a1-492f-bfb5-337c9ffc9dfc None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:34.047498 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:34.079903 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79bf69-0f40-4540-96da-219365b9194a None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:34.115695 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b345ffd-126c-4bd9-89ab-4c78f88dfb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:34.116366 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b345ffd-126c-4bd9-89ab-4c78f88dfb96 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:34.116664 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b345ffd-126c-4bd9-89ab-4c78f88dfb96 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:34.117172 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b345ffd-126c-4bd9-89ab-4c78f88dfb96 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:34.118410 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b345ffd-126c-4bd9-89ab-4c78f88dfb96 None admin] RBAC: Adding query filter params (name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:34.118850 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b345ffd-126c-4bd9-89ab-4c78f88dfb96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:34.120272 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b345ffd-126c-4bd9-89ab-4c78f88dfb96 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:34.128998 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 283/567] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Jun 15 14:29:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:34.129903 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b345ffd-126c-4bd9-89ab-4c78f88dfb96 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:34.130320 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83864000-e5a1-492f-bfb5-337c9ffc9dfc None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:34.165242 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb21287-22b2-46dc-8d87-3ec6693967d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:34.165831 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb21287-22b2-46dc-8d87-3ec6693967d1 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:34.166110 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb21287-22b2-46dc-8d87-3ec6693967d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:34.166383 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb21287-22b2-46dc-8d87-3ec6693967d1 None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:34.167240 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb21287-22b2-46dc-8d87-3ec6693967d1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:34.167549 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb21287-22b2-46dc-8d87-3ec6693967d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:34.168978 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb21287-22b2-46dc-8d87-3ec6693967d1 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:34.186789 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 285/568] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Jun 15 14:29:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:34.187624 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb21287-22b2-46dc-8d87-3ec6693967d1 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:34.192349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:34.230573 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b345ffd-126c-4bd9-89ab-4c78f88dfb96 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:34.264401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e698ae-2612-4fb8-a46e-0605d9fd2ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:34.265195 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e698ae-2612-4fb8-a46e-0605d9fd2ca4 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:34.265480 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e698ae-2612-4fb8-a46e-0605d9fd2ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:34.265756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e698ae-2612-4fb8-a46e-0605d9fd2ca4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:34.270529 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:34.276725 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e698ae-2612-4fb8-a46e-0605d9fd2ca4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:34.284849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-52e698ae-2612-4fb8-a46e-0605d9fd2ca4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:34.284849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:34.284849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:34.284849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:34.284849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:34.284849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:34.284849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:34.284849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:34.284849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:34.284849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:34.284849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:34.284849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:34.284849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:34.284849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:34.284849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:34.284849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:34.285912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:34.285912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:34.285912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:34.285912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:34.285912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:34.285912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:34.285912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:34.285912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:34.285912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:34.285912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:34.285912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:34.285912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:34.285912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:34.285912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:34.285912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:34.285912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:34.286912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:34.286912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:34.286912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:34.286912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:34.286912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:34.286912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:34.286912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:34.286912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:34.286912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:34.286912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:34.286912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:34.286912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:34.286912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 284/569] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Jun 15 14:29:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:34.287921 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c867d-1ad5-464c-ba7b-0cb157137b71 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:34.287921 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c867d-1ad5-464c-ba7b-0cb157137b71 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:34.287921 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c867d-1ad5-464c-ba7b-0cb157137b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:34.291919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb21287-22b2-46dc-8d87-3ec6693967d1 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:34.304507 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-b78c867d-1ad5-464c-ba7b-0cb157137b71 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:34.305703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b78c867d-1ad5-464c-ba7b-0cb157137b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:34.339320 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45563586-e62b-4329-96e7-4e33dd59dbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:34.339985 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45563586-e62b-4329-96e7-4e33dd59dbc8 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:34.340265 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45563586-e62b-4329-96e7-4e33dd59dbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:34.340537 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45563586-e62b-4329-96e7-4e33dd59dbc8 None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:34.341325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45563586-e62b-4329-96e7-4e33dd59dbc8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:34.341635 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45563586-e62b-4329-96e7-4e33dd59dbc8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:34.343121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45563586-e62b-4329-96e7-4e33dd59dbc8 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:34.354483 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 286/570] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:29:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:34.355413 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45563586-e62b-4329-96e7-4e33dd59dbc8 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:34.378385 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 285/571] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:34.379256 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c867d-1ad5-464c-ba7b-0cb157137b71 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:34.443558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:34.454704 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45563586-e62b-4329-96e7-4e33dd59dbc8 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:34.487945 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f8428-d352-4fd6-a461-54499c16cfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:34.488154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c867d-1ad5-464c-ba7b-0cb157137b71 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:34.488731 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f8428-d352-4fd6-a461-54499c16cfb8 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:34.489010 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f8428-d352-4fd6-a461-54499c16cfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:34.489280 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f8428-d352-4fd6-a461-54499c16cfb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:34.500088 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9f8428-d352-4fd6-a461-54499c16cfb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:34.501523 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9f8428-d352-4fd6-a461-54499c16cfb8 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:34.503251 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:34.519403 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-0e9f8428-d352-4fd6-a461-54499c16cfb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:34.519403 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:34.519403 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:34.519403 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:34.519403 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:34.519403 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:34.519403 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:34.519403 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:34.519403 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:34.519403 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:34.519403 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:34.519403 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:34.519403 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:34.519403 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:34.519403 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:34.520356 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:34.520356 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:34.520356 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:34.520356 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:34.520356 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:34.520356 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:34.520356 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:34.520356 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:34.520356 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:34.520356 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:34.520356 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:34.520356 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:34.520356 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:34.520356 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:34.520356 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:34.520356 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:34.520356 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:34.521269 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:34.521269 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:34.521269 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:34.521269 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:34.521269 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 287/572] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:34.521537 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f8428-d352-4fd6-a461-54499c16cfb8 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:34.530694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d21223d-ad71-45e5-9ba6-d140ccb0124b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:34.531520 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21223d-ad71-45e5-9ba6-d140ccb0124b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:34.531804 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21223d-ad71-45e5-9ba6-d140ccb0124b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:34.532079 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21223d-ad71-45e5-9ba6-d140ccb0124b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:34.542072 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d21223d-ad71-45e5-9ba6-d140ccb0124b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:34.544098 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d21223d-ad71-45e5-9ba6-d140ccb0124b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:34.562237 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-6d21223d-ad71-45e5-9ba6-d140ccb0124b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:34.562237 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:34.562237 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:34.562237 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:34.562237 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:34.562237 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:34.562237 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:34.562237 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:34.562237 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:34.562237 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:34.562237 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:34.562237 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:34.562237 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:34.562237 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:34.562237 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:34.563158 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:34.563158 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:34.563158 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:34.563158 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:34.563158 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:34.563158 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:34.563158 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:34.563158 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:34.563158 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:34.563158 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:34.563158 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:34.563158 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:34.563158 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:34.563158 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:34.563158 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:34.563158 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:34.563158 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:34.564042 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:34.564042 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:34.564042 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:34.564042 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:34.564042 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 286/573] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:34.564305 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d21223d-ad71-45e5-9ba6-d140ccb0124b None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:34.626312 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f8428-d352-4fd6-a461-54499c16cfb8 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:34.659726 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d2bde-284d-483c-9bdb-86fafffcade6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:34.660446 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d2bde-284d-483c-9bdb-86fafffcade6 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:34.660670 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d2bde-284d-483c-9bdb-86fafffcade6 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:34.660942 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d2bde-284d-483c-9bdb-86fafffcade6 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:34.661691 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893d2bde-284d-483c-9bdb-86fafffcade6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:34.662001 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893d2bde-284d-483c-9bdb-86fafffcade6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:34.662432 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d21223d-ad71-45e5-9ba6-d140ccb0124b None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:34.663410 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893d2bde-284d-483c-9bdb-86fafffcade6 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:34.670172 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:34.672688 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 288/574] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Jun 15 14:29:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:34.673336 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d2bde-284d-483c-9bdb-86fafffcade6 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:34.701630 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd26a5da-85cd-44b5-b160-ef41f2908a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:34.702222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26a5da-85cd-44b5-b160-ef41f2908a46 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:34.702508 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26a5da-85cd-44b5-b160-ef41f2908a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:34.702787 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26a5da-85cd-44b5-b160-ef41f2908a46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:34.703653 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd26a5da-85cd-44b5-b160-ef41f2908a46 None admin] RBAC: Adding query filter params (role.id=13dcf68cca9c44888c0fd27a78db3dbe, scope.project.id=d488c79cb1704e9496fc196d8aa6c104, user.id=641a81b3140f47119b5c2323a97a6681) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:34.703969 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd26a5da-85cd-44b5-b160-ef41f2908a46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:34.705401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd26a5da-85cd-44b5-b160-ef41f2908a46 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:34.710898 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 287/575] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:29:34 2022] GET /identity/v3/role_assignments?user.id=641a81b3140f47119b5c2323a97a6681&scope.project.id=d488c79cb1704e9496fc196d8aa6c104&role.id=13dcf68cca9c44888c0fd27a78db3dbe => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:34.711707 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd26a5da-85cd-44b5-b160-ef41f2908a46 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:34.752994 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:34.803762 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d2bde-284d-483c-9bdb-86fafffcade6 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:34.832279 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd26a5da-85cd-44b5-b160-ef41f2908a46 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:34.843433 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4aa5f1-0dd2-41ed-bceb-1707f855b5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:34.844258 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4aa5f1-0dd2-41ed-bceb-1707f855b5bd None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:34.844538 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4aa5f1-0dd2-41ed-bceb-1707f855b5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:34.844818 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4aa5f1-0dd2-41ed-bceb-1707f855b5bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:34.856779 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4aa5f1-0dd2-41ed-bceb-1707f855b5bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:34.864601 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-eb4aa5f1-0dd2-41ed-bceb-1707f855b5bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:34.864601 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:34.864601 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:34.864601 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:34.864601 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:34.864601 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:34.864601 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:34.864601 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:34.864601 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:34.864601 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:34.864601 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:34.864601 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:34.864601 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:34.864601 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:34.864601 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:34.864601 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:34.865604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:34.865604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:34.865604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:34.865604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:34.865604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:34.865604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:34.865604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:34.865604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:34.865604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:34.865604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:34.865604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:34.865604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:34.865604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:34.865604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:34.865604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:34.865604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:34.871845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:34.871845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:34.871845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:34.871845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:34.871845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:34.871845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:34.871845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:34.871845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:34.871845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:34.871845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:34.871845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:34.871845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:34.871845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 289/576] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Jun 15 14:29:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:34.871845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4aa5f1-0dd2-41ed-bceb-1707f855b5bd None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:34.876830 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53e52e-8979-48cb-bf8d-0db3f0f290d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:34.877494 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53e52e-8979-48cb-bf8d-0db3f0f290d4 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:34.877964 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53e52e-8979-48cb-bf8d-0db3f0f290d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:34.878078 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53e52e-8979-48cb-bf8d-0db3f0f290d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:34.878756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d53e52e-8979-48cb-bf8d-0db3f0f290d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:34.879090 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d53e52e-8979-48cb-bf8d-0db3f0f290d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:34.880514 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d53e52e-8979-48cb-bf8d-0db3f0f290d4 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:34.895240 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 288/577] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:34.900361 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53e52e-8979-48cb-bf8d-0db3f0f290d4 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:34.932429 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:34.982904 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4aa5f1-0dd2-41ed-bceb-1707f855b5bd None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:35.001057 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:35.005918 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53e52e-8979-48cb-bf8d-0db3f0f290d4 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:35.021031 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a27df9-c4b9-48eb-8058-570181a639e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:35.021698 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a27df9-c4b9-48eb-8058-570181a639e6 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:35.022015 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a27df9-c4b9-48eb-8058-570181a639e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:35.022466 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a27df9-c4b9-48eb-8058-570181a639e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:35.023162 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a27df9-c4b9-48eb-8058-570181a639e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:35.023470 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a27df9-c4b9-48eb-8058-570181a639e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:35.024856 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a27df9-c4b9-48eb-8058-570181a639e6 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:35.043844 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 290/578] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:35.043844 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a27df9-c4b9-48eb-8058-570181a639e6 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:35.047249 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c0288-bc08-43c2-b81c-5e716b3b458b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:35.048049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c0288-bc08-43c2-b81c-5e716b3b458b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:35.048329 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c0288-bc08-43c2-b81c-5e716b3b458b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:35.048602 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c0288-bc08-43c2-b81c-5e716b3b458b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:35.059060 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0c0288-bc08-43c2-b81c-5e716b3b458b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:35.060512 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0c0288-bc08-43c2-b81c-5e716b3b458b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:35.078978 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-6a0c0288-bc08-43c2-b81c-5e716b3b458b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:35.078978 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:35.078978 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:35.078978 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:35.078978 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:35.078978 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:35.078978 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:35.078978 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:35.078978 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:35.078978 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:35.078978 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:35.078978 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:35.078978 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:35.078978 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:35.078978 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:35.079880 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:35.079880 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:35.079880 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:35.079880 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:35.079880 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:35.079880 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:35.079880 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:35.079880 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:35.079880 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:35.079880 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:35.079880 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:35.079880 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:35.079880 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:35.079880 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:35.079880 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:35.079880 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:35.079880 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:35.080782 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:35.080782 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:35.080782 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:35.080782 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:35.080782 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 289/579] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:35.081057 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c0288-bc08-43c2-b81c-5e716b3b458b None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:35.157204 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a27df9-c4b9-48eb-8058-570181a639e6 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:35.181499 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:35.196644 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4eaf17-27fa-4368-9905-40549c36346e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:35.197323 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4eaf17-27fa-4368-9905-40549c36346e None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:35.197615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4eaf17-27fa-4368-9905-40549c36346e None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:35.197893 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4eaf17-27fa-4368-9905-40549c36346e None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:35.199311 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4eaf17-27fa-4368-9905-40549c36346e None admin] RBAC: Adding query filter params (name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:35.199484 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4eaf17-27fa-4368-9905-40549c36346e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:35.200845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4eaf17-27fa-4368-9905-40549c36346e None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:35.202230 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c0288-bc08-43c2-b81c-5e716b3b458b None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:35.213480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 291/580] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Jun 15 14:29:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:35.213480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4eaf17-27fa-4368-9905-40549c36346e None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:35.243281 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df15fa26-a51f-4a9f-9259-00829b44b951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:35.244308 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15fa26-a51f-4a9f-9259-00829b44b951 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:35.244594 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15fa26-a51f-4a9f-9259-00829b44b951 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:35.244874 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15fa26-a51f-4a9f-9259-00829b44b951 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:35.252160 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df15fa26-a51f-4a9f-9259-00829b44b951 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:35.253598 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df15fa26-a51f-4a9f-9259-00829b44b951 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:35.257620 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:35.269842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-df15fa26-a51f-4a9f-9259-00829b44b951 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 15 14:29:35.269842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:35.269842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:35.269842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:35.269842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:35.269842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:35.269842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:35.269842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:35.269842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:35.269842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:35.269842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:35.269842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:35.269842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:29:35.269842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:29:35.269842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:29:35.269842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:29:35.270799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:35.270799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:35.270799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:29:35.270799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:35.270799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:29:35.270799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:35.270799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:35.270799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:35.270799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:35.270799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:35.270799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:35.270799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:35.270799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:35.270799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:35.270799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:35.270799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:35.270799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:35.270799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:35.272431 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:35.272431 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:35.272431 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:29:35.272431 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:29:35.272431 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:29:35.272431 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:29:35.272431 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:29:35.272431 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:29:35.272431 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 15 14:29:35.272431 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:35.272431 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 290/581] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:35 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:35.272431 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df15fa26-a51f-4a9f-9259-00829b44b951 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:35.325283 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4eaf17-27fa-4368-9905-40549c36346e None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:35.358872 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942da6fe-e1dd-45ee-b3d4-55cec1998c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:35.359699 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942da6fe-e1dd-45ee-b3d4-55cec1998c1b None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:35.359985 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942da6fe-e1dd-45ee-b3d4-55cec1998c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:35.360262 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942da6fe-e1dd-45ee-b3d4-55cec1998c1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:35.367136 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942da6fe-e1dd-45ee-b3d4-55cec1998c1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:35.368591 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942da6fe-e1dd-45ee-b3d4-55cec1998c1b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:35.376775 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df15fa26-a51f-4a9f-9259-00829b44b951 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:35.382529 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-942da6fe-e1dd-45ee-b3d4-55cec1998c1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:35.382529 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:35.382529 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:35.382529 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:35.382529 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:35.382529 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:35.382529 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:35.382529 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:35.382529 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:35.382529 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:35.382529 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:35.382529 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:35.382529 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:29:35.382529 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:29:35.382529 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:29:35.383431 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:29:35.383431 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:35.383431 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:35.383431 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:35.383431 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:35.383431 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:35.383431 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:35.383431 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:35.383431 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:35.383431 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:35.383431 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:35.383431 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:35.383431 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:35.383431 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:35.383431 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:35.383431 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:35.383431 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:35.384317 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:29:35.384317 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:29:35.384317 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:29:35.384317 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:29:35.384317 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:29:35.384317 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:29:35.384317 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:35.384317 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:35.384317 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 292/582] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:35.385392 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942da6fe-e1dd-45ee-b3d4-55cec1998c1b None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:35.420795 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49c15a7-aa01-4626-a641-79cc472401f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:35.421467 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c15a7-aa01-4626-a641-79cc472401f3 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:35.421753 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c15a7-aa01-4626-a641-79cc472401f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:35.422031 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c15a7-aa01-4626-a641-79cc472401f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:35.422764 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49c15a7-aa01-4626-a641-79cc472401f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:35.423108 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49c15a7-aa01-4626-a641-79cc472401f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:35.424490 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49c15a7-aa01-4626-a641-79cc472401f3 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:35.438762 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 291/583] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:35.440034 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49c15a7-aa01-4626-a641-79cc472401f3 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:35.445692 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:35.493098 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:35.497825 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942da6fe-e1dd-45ee-b3d4-55cec1998c1b None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:35.539737 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0713d32-3f22-4e85-87ab-3b2fe26f7a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:35.540927 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0713d32-3f22-4e85-87ab-3b2fe26f7a66 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:35.541479 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0713d32-3f22-4e85-87ab-3b2fe26f7a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:35.541760 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0713d32-3f22-4e85-87ab-3b2fe26f7a66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:35.549038 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49c15a7-aa01-4626-a641-79cc472401f3 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:35.555065 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0713d32-3f22-4e85-87ab-3b2fe26f7a66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:35.556617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0713d32-3f22-4e85-87ab-3b2fe26f7a66 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:35.576302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-d0713d32-3f22-4e85-87ab-3b2fe26f7a66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:35.576302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:35.576302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:35.576302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:35.576302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:35.576302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:35.576302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:35.576302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:35.576302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:35.576302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:35.576302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:35.576302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:35.576302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:35.576302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:35.576302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:35.577168 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:35.577168 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:35.577168 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:35.577168 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:35.577168 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:35.577168 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:35.577168 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:35.577168 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:35.577168 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:35.577168 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:35.577168 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:35.577168 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:35.577168 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:35.577168 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:35.577168 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:35.577168 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:35.577168 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:35.578018 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:35.578018 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:35.578018 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:35.578018 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:35.578018 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 293/584] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:35.578809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0713d32-3f22-4e85-87ab-3b2fe26f7a66 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:35.587595 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b63f0-7ae2-48c3-9237-587c841feddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:35.588269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b63f0-7ae2-48c3-9237-587c841feddf None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:35.588554 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b63f0-7ae2-48c3-9237-587c841feddf None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:35.588838 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b63f0-7ae2-48c3-9237-587c841feddf None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:35.589703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b63f0-7ae2-48c3-9237-587c841feddf None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:35.590013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b63f0-7ae2-48c3-9237-587c841feddf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:35.591496 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b63f0-7ae2-48c3-9237-587c841feddf None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:35.612654 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 292/585] 10.222.0.45 () {56 vars in 1137 bytes} [Wed Jun 15 14:29:35 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:35.612654 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b63f0-7ae2-48c3-9237-587c841feddf None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:35.674062 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:35.689533 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0713d32-3f22-4e85-87ab-3b2fe26f7a66 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:35.724751 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b63f0-7ae2-48c3-9237-587c841feddf None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:35.736794 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b7f599-81a0-4227-abe1-abc1d265eb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:35.737778 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7f599-81a0-4227-abe1-abc1d265eb21 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:35.738211 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7f599-81a0-4227-abe1-abc1d265eb21 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:35.738774 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7f599-81a0-4227-abe1-abc1d265eb21 None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:35.740111 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b7f599-81a0-4227-abe1-abc1d265eb21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:35.740831 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b7f599-81a0-4227-abe1-abc1d265eb21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:35.742872 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b7f599-81a0-4227-abe1-abc1d265eb21 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:35.762423 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 294/586] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:29:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:35.762423 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b7f599-81a0-4227-abe1-abc1d265eb21 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:35.764267 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524d73a-5b31-4070-84f1-73a1ec6cf121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:35.765159 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524d73a-5b31-4070-84f1-73a1ec6cf121 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:35.765445 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524d73a-5b31-4070-84f1-73a1ec6cf121 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:35.765716 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524d73a-5b31-4070-84f1-73a1ec6cf121 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:35.773233 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:35.774663 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5524d73a-5b31-4070-84f1-73a1ec6cf121 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:35.776513 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5524d73a-5b31-4070-84f1-73a1ec6cf121 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:35.793814 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-5524d73a-5b31-4070-84f1-73a1ec6cf121 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 14:29:35.793814 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:35.793814 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:35.793814 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:35.793814 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:35.793814 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:35.793814 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:35.793814 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:35.793814 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:35.793814 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:35.793814 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:35.793814 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:35.793814 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:29:35.793814 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:29:35.793814 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:29:35.793814 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:29:35.794967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:35.794967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:35.794967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:29:35.794967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:35.794967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:29:35.794967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:35.794967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:35.794967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:35.794967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:35.794967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:35.794967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:35.794967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:35.794967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:35.794967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:35.794967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:35.794967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:35.794967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:35.794967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:35.796077 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:35.796077 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:35.796077 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:29:35.796077 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:29:35.796077 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:29:35.796077 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:29:35.796077 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:29:35.796077 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:29:35.796077 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 14:29:35.796077 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:35.796077 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 293/587] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Jun 15 14:29:35 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:35.796652 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524d73a-5b31-4070-84f1-73a1ec6cf121 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:35.878682 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b7f599-81a0-4227-abe1-abc1d265eb21 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:35.907006 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524d73a-5b31-4070-84f1-73a1ec6cf121 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:35.915459 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad06fb-8c12-4957-bf0f-cea4636a03b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:35.916304 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad06fb-8c12-4957-bf0f-cea4636a03b3 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:35.916738 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad06fb-8c12-4957-bf0f-cea4636a03b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:35.917007 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad06fb-8c12-4957-bf0f-cea4636a03b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:35.917724 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ad06fb-8c12-4957-bf0f-cea4636a03b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:35.918025 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ad06fb-8c12-4957-bf0f-cea4636a03b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:35.919423 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ad06fb-8c12-4957-bf0f-cea4636a03b3 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:35.928983 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:35.936228 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 295/588] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:35.936228 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad06fb-8c12-4957-bf0f-cea4636a03b3 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:35.948377 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a4ab0-7af4-4ccd-b9fa-02d794713ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:35.949714 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a4ab0-7af4-4ccd-b9fa-02d794713ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:35.949997 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a4ab0-7af4-4ccd-b9fa-02d794713ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:35.950271 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a4ab0-7af4-4ccd-b9fa-02d794713ea8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:35.960354 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a4ab0-7af4-4ccd-b9fa-02d794713ea8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:35.961791 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a4ab0-7af4-4ccd-b9fa-02d794713ea8 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:35.980150 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-e49a4ab0-7af4-4ccd-b9fa-02d794713ea8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:35.980150 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:35.980150 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:35.980150 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:35.980150 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:35.980150 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:35.980150 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:35.980150 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:35.980150 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:35.980150 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:35.980150 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:35.980150 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:35.980150 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:35.980150 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:35.980150 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:35.981027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:35.981027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:35.981027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:35.981027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:35.981027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:35.981027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:35.981027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:35.981027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:35.981027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:35.981027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:35.981027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:35.981027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:35.981027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:35.981027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:35.981027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:35.981027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:35.981027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:35.981991 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:35.981991 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:35.981991 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:35.981991 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:35.981991 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 294/589] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:35.982272 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a4ab0-7af4-4ccd-b9fa-02d794713ea8 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:36.019859 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:36.051250 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad06fb-8c12-4957-bf0f-cea4636a03b3 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:36.085498 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014db916-b860-458a-a17d-5a934d34f64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:36.086381 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014db916-b860-458a-a17d-5a934d34f64e None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:36.086671 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014db916-b860-458a-a17d-5a934d34f64e None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:36.086946 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014db916-b860-458a-a17d-5a934d34f64e None admin] PATH_INFO: `/v3/projects/d488c79cb1704e9496fc196d8aa6c104/users/f3185bd2458c4e22830975700888e9be/roles/13dcf68cca9c44888c0fd27a78db3dbe` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:36.087759 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a4ab0-7af4-4ccd-b9fa-02d794713ea8 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:36.102090 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014db916-b860-458a-a17d-5a934d34f64e None admin] RBAC: Authorizing `identity:create_grant(project_id=d488c79cb1704e9496fc196d8aa6c104, user_id=f3185bd2458c4e22830975700888e9be, role_id=13dcf68cca9c44888c0fd27a78db3dbe)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:36.103575 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014db916-b860-458a-a17d-5a934d34f64e None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:36.117297 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 296/590] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Jun 15 14:29:35 2022] PUT /identity/v3/projects/d488c79cb1704e9496fc196d8aa6c104/users/f3185bd2458c4e22830975700888e9be/roles/13dcf68cca9c44888c0fd27a78db3dbe => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:29:36.118086 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014db916-b860-458a-a17d-5a934d34f64e None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:36.137244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d4a2b-970d-4c74-9e18-ae3cb7aee6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:36.138418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d4a2b-970d-4c74-9e18-ae3cb7aee6b9 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:36.138902 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d4a2b-970d-4c74-9e18-ae3cb7aee6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:36.139375 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d4a2b-970d-4c74-9e18-ae3cb7aee6b9 None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:36.140593 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034d4a2b-970d-4c74-9e18-ae3cb7aee6b9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:36.141098 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034d4a2b-970d-4c74-9e18-ae3cb7aee6b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:36.142888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034d4a2b-970d-4c74-9e18-ae3cb7aee6b9 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:36.170669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 295/591] 10.222.0.45 () {56 vars in 1139 bytes} [Wed Jun 15 14:29:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:36.171855 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d4a2b-970d-4c74-9e18-ae3cb7aee6b9 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:36.211442 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:36.263066 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014db916-b860-458a-a17d-5a934d34f64e None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:36.290739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:36.300708 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d4a2b-970d-4c74-9e18-ae3cb7aee6b9 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:36.318404 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cad49e-426a-433b-8809-70cac90d7d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:36.319607 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cad49e-426a-433b-8809-70cac90d7d3c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:36.320091 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cad49e-426a-433b-8809-70cac90d7d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:36.320562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cad49e-426a-433b-8809-70cac90d7d3c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:36.329985 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cad49e-426a-433b-8809-70cac90d7d3c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:36.331968 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cad49e-426a-433b-8809-70cac90d7d3c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:36.342529 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e5b562-06e6-49d2-9fad-a6e4129027a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:36.343225 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5b562-06e6-49d2-9fad-a6e4129027a7 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:36.343508 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5b562-06e6-49d2-9fad-a6e4129027a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:36.343783 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5b562-06e6-49d2-9fad-a6e4129027a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:36.344502 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e5b562-06e6-49d2-9fad-a6e4129027a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:36.344809 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e5b562-06e6-49d2-9fad-a6e4129027a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:36.346167 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e5b562-06e6-49d2-9fad-a6e4129027a7 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:36.349474 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-b1cad49e-426a-433b-8809-70cac90d7d3c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 14:29:36.349474 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:36.349474 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:36.349474 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:36.349474 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:36.349474 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:36.349474 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:36.349474 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:36.349474 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:36.349474 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:36.349474 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:36.349474 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:36.349474 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:29:36.349474 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:29:36.349474 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:29:36.349474 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:29:36.350384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:36.350384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:36.350384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:29:36.350384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:36.350384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:29:36.350384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:36.350384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:36.350384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:36.350384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:36.350384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:36.350384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:36.350384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:36.350384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:36.350384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:36.350384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:36.350384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:36.350384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:36.350384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:36.351327 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:36.351327 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:36.351327 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:29:36.351327 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:29:36.351327 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:29:36.351327 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:29:36.351327 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:29:36.351327 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:29:36.351327 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 14:29:36.351327 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:36.353650 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 297/592] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:36.354818 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cad49e-426a-433b-8809-70cac90d7d3c None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:36.358277 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 296/593] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:36.363303 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e5b562-06e6-49d2-9fad-a6e4129027a7 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:36.471238 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cad49e-426a-433b-8809-70cac90d7d3c None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:36.493794 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:36.508066 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e5b562-06e6-49d2-9fad-a6e4129027a7 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:36.509426 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0744e-cb8d-4796-b86a-ee28780096e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:36.510395 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0744e-cb8d-4796-b86a-ee28780096e5 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:36.510821 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0744e-cb8d-4796-b86a-ee28780096e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:36.511870 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0744e-cb8d-4796-b86a-ee28780096e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:36.524812 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f0744e-cb8d-4796-b86a-ee28780096e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:36.526502 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f0744e-cb8d-4796-b86a-ee28780096e5 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:36.546348 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-84f0744e-cb8d-4796-b86a-ee28780096e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:36.546348 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:36.546348 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:36.546348 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:36.546348 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:36.546348 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:36.546348 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:36.546348 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:36.546348 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:36.546348 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:36.546348 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:36.546348 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:36.546348 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:36.546348 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:36.546348 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:36.547444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:36.547444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:36.547444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:36.547444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:36.547444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:36.547444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:36.547444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:36.547444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:36.547444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:36.547444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:36.547444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:36.547444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:36.547444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:36.547444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:36.547444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:36.547444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:36.547444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:36.548466 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:36.548466 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:36.548466 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:36.548466 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:36.554426 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a710e2-3a97-4cc7-a104-579e515e0c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:36.554426 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a710e2-3a97-4cc7-a104-579e515e0c8c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:36.554426 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a710e2-3a97-4cc7-a104-579e515e0c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:36.554426 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a710e2-3a97-4cc7-a104-579e515e0c8c None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:36.554864 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 298/594] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:36.555395 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a710e2-3a97-4cc7-a104-579e515e0c8c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:36.555789 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a710e2-3a97-4cc7-a104-579e515e0c8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:36.556206 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0744e-cb8d-4796-b86a-ee28780096e5 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:36.557356 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a710e2-3a97-4cc7-a104-579e515e0c8c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:36.579401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 297/595] 10.222.0.45 () {56 vars in 1137 bytes} [Wed Jun 15 14:29:36 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:36.580507 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a710e2-3a97-4cc7-a104-579e515e0c8c None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:36.605546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:36.694402 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0744e-cb8d-4796-b86a-ee28780096e5 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:36.698436 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a710e2-3a97-4cc7-a104-579e515e0c8c None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:36.741291 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb1f76d-f70b-4c95-b71d-906d95aa7c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:36.741728 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f231c-30be-4dae-84b7-7d63f5dfc462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:36.742091 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1f76d-f70b-4c95-b71d-906d95aa7c13 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:36.742375 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1f76d-f70b-4c95-b71d-906d95aa7c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:36.742707 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1f76d-f70b-4c95-b71d-906d95aa7c13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:36.743367 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f231c-30be-4dae-84b7-7d63f5dfc462 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:36.743846 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f231c-30be-4dae-84b7-7d63f5dfc462 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:36.744312 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f231c-30be-4dae-84b7-7d63f5dfc462 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:36.745226 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f231c-30be-4dae-84b7-7d63f5dfc462 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:36.746302 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f231c-30be-4dae-84b7-7d63f5dfc462 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:36.747919 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f231c-30be-4dae-84b7-7d63f5dfc462 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:36.751218 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb1f76d-f70b-4c95-b71d-906d95aa7c13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:36.752665 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb1f76d-f70b-4c95-b71d-906d95aa7c13 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:36.760688 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 298/596] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:36.761789 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f231c-30be-4dae-84b7-7d63f5dfc462 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:36.770021 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-8fb1f76d-f70b-4c95-b71d-906d95aa7c13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:36.770021 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:36.770021 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:36.770021 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:36.770021 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:36.770021 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:36.770021 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:36.770021 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:36.770021 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:36.770021 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:36.770021 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:36.770021 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:36.770021 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:29:36.770021 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:29:36.770021 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:29:36.770896 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:29:36.770896 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:36.770896 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:36.770896 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:36.770896 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:36.770896 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:36.770896 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:36.770896 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:36.770896 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:36.770896 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:36.770896 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:36.770896 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:36.770896 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:36.770896 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:36.770896 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:36.770896 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:36.770896 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:36.771801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:29:36.771801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:29:36.771801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:29:36.771801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:29:36.771801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:29:36.771801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:29:36.771801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:36.771801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:36.775062 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 299/597] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:36.776100 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb1f76d-f70b-4c95-b71d-906d95aa7c13 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:36.779040 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:36.896055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:36.898701 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f231c-30be-4dae-84b7-7d63f5dfc462 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:36.913384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb1f76d-f70b-4c95-b71d-906d95aa7c13 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:36.940580 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358f983-e5a8-4364-b9ac-82c7eb5b2df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:36.941812 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358f983-e5a8-4364-b9ac-82c7eb5b2df3 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:36.942359 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358f983-e5a8-4364-b9ac-82c7eb5b2df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:36.942640 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358f983-e5a8-4364-b9ac-82c7eb5b2df3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:36.948799 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96046f4-cbc5-484f-b0c7-a92df6d84566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:36.949590 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96046f4-cbc5-484f-b0c7-a92df6d84566 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:36.949869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96046f4-cbc5-484f-b0c7-a92df6d84566 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:36.950141 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96046f4-cbc5-484f-b0c7-a92df6d84566 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:36.955507 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358f983-e5a8-4364-b9ac-82c7eb5b2df3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:36.957056 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96046f4-cbc5-484f-b0c7-a92df6d84566 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:36.957233 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358f983-e5a8-4364-b9ac-82c7eb5b2df3 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:36.958486 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96046f4-cbc5-484f-b0c7-a92df6d84566 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:36.972562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-b96046f4-cbc5-484f-b0c7-a92df6d84566 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:36.972562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:36.972562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:36.972562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:36.972562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:36.972562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:36.972562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:36.972562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:36.972562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:36.972562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:36.972562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:36.972562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:36.972562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:29:36.972562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:29:36.972562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:29:36.979507 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:29:36.979507 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:36.979507 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:36.979507 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:36.979507 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:36.979507 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:36.979507 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:36.979507 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:36.979507 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:36.979507 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:36.979507 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:36.979507 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:36.979507 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:36.979507 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:36.979507 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:36.979507 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:36.979507 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:36.986338 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:29:36.986338 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:29:36.986338 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:29:36.986338 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:29:36.986338 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:29:36.986338 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:29:36.986338 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:36.986338 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:36.986338 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 300/598] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:36.986338 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96046f4-cbc5-484f-b0c7-a92df6d84566 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:36.988658 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-1358f983-e5a8-4364-b9ac-82c7eb5b2df3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:36.988658 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:36.988658 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:36.988658 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:36.988658 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:36.988658 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:36.988658 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:36.988658 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:36.988658 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:36.988658 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:36.988658 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:36.988658 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:36.988658 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:36.988658 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:36.988658 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:36.990506 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:36.990506 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:36.990506 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:36.990506 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:36.990506 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:36.990506 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:36.990506 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:36.990506 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:36.990506 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:36.990506 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:36.990506 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:36.990506 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:36.990506 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:36.990506 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:36.990506 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:36.990506 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:36.990506 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:36.992354 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:36.992354 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:36.992354 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:36.992354 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:36.992354 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 299/599] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:36.992354 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46106775-2dde-41d1-9748-cf6d647bfd1c None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:36.992354 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46106775-2dde-41d1-9748-cf6d647bfd1c None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:36.992354 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46106775-2dde-41d1-9748-cf6d647bfd1c None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:36.992354 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 300/600] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:36.993829 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46106775-2dde-41d1-9748-cf6d647bfd1c None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:37.049655 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:37.090278 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96046f4-cbc5-484f-b0c7-a92df6d84566 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:37.092167 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46106775-2dde-41d1-9748-cf6d647bfd1c None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:37.125797 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920a98f-aba3-4246-820f-1254347d1c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:37.126241 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d36b5-b0ab-4168-abbb-ae1dab2b1bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:37.126455 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920a98f-aba3-4246-820f-1254347d1c2d None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:37.126734 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920a98f-aba3-4246-820f-1254347d1c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:37.126918 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d36b5-b0ab-4168-abbb-ae1dab2b1bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:37.127109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920a98f-aba3-4246-820f-1254347d1c2d None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:37.127205 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d36b5-b0ab-4168-abbb-ae1dab2b1bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:37.127471 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d36b5-b0ab-4168-abbb-ae1dab2b1bf4 None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:37.127801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920a98f-aba3-4246-820f-1254347d1c2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:37.128110 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920a98f-aba3-4246-820f-1254347d1c2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:37.128262 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d36b5-b0ab-4168-abbb-ae1dab2b1bf4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:37.128610 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d36b5-b0ab-4168-abbb-ae1dab2b1bf4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:37.129553 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920a98f-aba3-4246-820f-1254347d1c2d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:37.130025 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d36b5-b0ab-4168-abbb-ae1dab2b1bf4 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:37.141228 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 301/601] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:29:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:37.147331 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 301/602] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:29:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:37.147331 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178acff-0de0-4049-aee1-61f66f31bf71 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:37.147331 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178acff-0de0-4049-aee1-61f66f31bf71 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:37.147331 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178acff-0de0-4049-aee1-61f66f31bf71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:37.147674 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920a98f-aba3-4246-820f-1254347d1c2d None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:37.162697 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:37.164896 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-5178acff-0de0-4049-aee1-61f66f31bf71 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:37.166091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-5178acff-0de0-4049-aee1-61f66f31bf71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:37.236755 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 302/603] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:37.237714 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5178acff-0de0-4049-aee1-61f66f31bf71 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:37.263235 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920a98f-aba3-4246-820f-1254347d1c2d None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:37.298796 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1074adc4-9a66-4eae-8e6a-05201a4a6c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:37.299755 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074adc4-9a66-4eae-8e6a-05201a4a6c09 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:37.300032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074adc4-9a66-4eae-8e6a-05201a4a6c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:37.300311 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074adc4-9a66-4eae-8e6a-05201a4a6c09 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:37.301332 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1074adc4-9a66-4eae-8e6a-05201a4a6c09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:37.301332 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1074adc4-9a66-4eae-8e6a-05201a4a6c09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:37.302699 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1074adc4-9a66-4eae-8e6a-05201a4a6c09 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:37.305702 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:37.314711 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 302/604] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:37.315559 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1074adc4-9a66-4eae-8e6a-05201a4a6c09 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:37.367932 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5178acff-0de0-4049-aee1-61f66f31bf71 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:37.388634 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:37.405374 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3a66b-6eae-4fef-8e6e-3ecb2fd1a5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:37.406185 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3a66b-6eae-4fef-8e6e-3ecb2fd1a5de None admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:37.406463 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3a66b-6eae-4fef-8e6e-3ecb2fd1a5de None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:37.406730 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3a66b-6eae-4fef-8e6e-3ecb2fd1a5de None admin] PATH_INFO: `/v3/projects/d488c79cb1704e9496fc196d8aa6c104/users/e4f10214b2bb4eafb3dff7adf71b9bf9/roles/13dcf68cca9c44888c0fd27a78db3dbe` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:37.420819 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1074adc4-9a66-4eae-8e6a-05201a4a6c09 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:37.421827 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3a66b-6eae-4fef-8e6e-3ecb2fd1a5de None admin] RBAC: Authorizing `identity:create_grant(project_id=d488c79cb1704e9496fc196d8aa6c104, user_id=e4f10214b2bb4eafb3dff7adf71b9bf9, role_id=13dcf68cca9c44888c0fd27a78db3dbe)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:37.424219 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3a66b-6eae-4fef-8e6e-3ecb2fd1a5de None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:37.441744 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 303/605] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Jun 15 14:29:37 2022] PUT /identity/v3/projects/d488c79cb1704e9496fc196d8aa6c104/users/e4f10214b2bb4eafb3dff7adf71b9bf9/roles/13dcf68cca9c44888c0fd27a78db3dbe => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:29:37.442124 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3a66b-6eae-4fef-8e6e-3ecb2fd1a5de None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:37.485186 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a94b6-51c3-4003-b99c-c9fa2ce81998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:37.486143 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a94b6-51c3-4003-b99c-c9fa2ce81998 None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:37.486429 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a94b6-51c3-4003-b99c-c9fa2ce81998 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:37.486706 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a94b6-51c3-4003-b99c-c9fa2ce81998 None admin] PATH_INFO: `/v3/projects/d488c79cb1704e9496fc196d8aa6c104/users/975ae2790ac849ff88e991231671f510/roles/13dcf68cca9c44888c0fd27a78db3dbe` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:37.526061 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a94b6-51c3-4003-b99c-c9fa2ce81998 None admin] RBAC: Authorizing `identity:create_grant(project_id=d488c79cb1704e9496fc196d8aa6c104, user_id=975ae2790ac849ff88e991231671f510, role_id=13dcf68cca9c44888c0fd27a78db3dbe)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:37.529420 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a94b6-51c3-4003-b99c-c9fa2ce81998 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:37.563324 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 303/606] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Jun 15 14:29:37 2022] PUT /identity/v3/projects/d488c79cb1704e9496fc196d8aa6c104/users/975ae2790ac849ff88e991231671f510/roles/13dcf68cca9c44888c0fd27a78db3dbe => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:29:37.565693 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a94b6-51c3-4003-b99c-c9fa2ce81998 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:37.612198 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:37.618129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3a66b-6eae-4fef-8e6e-3ecb2fd1a5de None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:37.668534 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75865cc1-eb8d-4a3b-ad43-6856eba9811f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:37.669714 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75865cc1-eb8d-4a3b-ad43-6856eba9811f None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:37.670242 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75865cc1-eb8d-4a3b-ad43-6856eba9811f None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:37.670739 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75865cc1-eb8d-4a3b-ad43-6856eba9811f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:37.684032 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75865cc1-eb8d-4a3b-ad43-6856eba9811f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:37.692565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-75865cc1-eb8d-4a3b-ad43-6856eba9811f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:37.692565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:37.692565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:37.692565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:37.692565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:37.692565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:37.692565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:37.692565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:37.692565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:37.692565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:37.692565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:37.692565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:37.692565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:37.692565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:37.692565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:37.692565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:37.693571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:37.693571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:37.693571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:37.693571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:37.693571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:37.693571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:37.693571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:37.693571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:37.693571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:37.693571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:37.693571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:37.693571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:37.693571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:37.693571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:37.693571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:37.693571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:37.694423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:37.694423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:37.694423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:37.694423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:37.694423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:37.694423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:37.694423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:37.694423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:37.694423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:37.694423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:37.694423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:37.694423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:37.697282 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 304/607] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Jun 15 14:29:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:37.704794 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75865cc1-eb8d-4a3b-ad43-6856eba9811f None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:37.736269 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a94b6-51c3-4003-b99c-c9fa2ce81998 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:37.767910 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:37.777475 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06af742-36b8-4f03-8fab-47bf22881503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:37.778310 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06af742-36b8-4f03-8fab-47bf22881503 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:37.778759 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06af742-36b8-4f03-8fab-47bf22881503 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:37.778985 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06af742-36b8-4f03-8fab-47bf22881503 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:37.787130 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06af742-36b8-4f03-8fab-47bf22881503 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:37.789013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06af742-36b8-4f03-8fab-47bf22881503 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:37.804672 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-f06af742-36b8-4f03-8fab-47bf22881503 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:37.804672 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:37.804672 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:37.804672 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:37.804672 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:37.804672 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:37.804672 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:37.804672 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:37.804672 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:37.804672 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:37.804672 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:37.804672 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:37.804672 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:29:37.804672 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:29:37.804672 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:29:37.805612 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:29:37.805612 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:37.805612 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:37.805612 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:37.805612 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:37.805612 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:37.805612 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:37.805612 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:37.805612 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:37.805612 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:37.805612 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:37.805612 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:37.805612 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:37.805612 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:37.805612 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:37.805612 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:37.805612 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:37.806546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:29:37.806546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:29:37.806546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:29:37.806546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:29:37.806546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:29:37.806546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:29:37.806546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:37.806546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:37.806546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 304/608] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:37.820832 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06af742-36b8-4f03-8fab-47bf22881503 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:37.843692 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75865cc1-eb8d-4a3b-ad43-6856eba9811f None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:37.889014 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7934c97-972c-449c-9566-266b77709625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:37.889978 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7934c97-972c-449c-9566-266b77709625 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:37.890265 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7934c97-972c-449c-9566-266b77709625 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:37.890694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7934c97-972c-449c-9566-266b77709625 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:37.891852 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7934c97-972c-449c-9566-266b77709625 None admin] RBAC: Adding query filter params (name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:37.892422 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7934c97-972c-449c-9566-266b77709625 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:37.893998 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7934c97-972c-449c-9566-266b77709625 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:37.909921 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 305/609] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Jun 15 14:29:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:37.915502 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7934c97-972c-449c-9566-266b77709625 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:37.942505 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06af742-36b8-4f03-8fab-47bf22881503 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:37.947226 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:37.997616 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d275900d-8ef6-4a24-af03-c72d18b2ea9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:37.998553 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275900d-8ef6-4a24-af03-c72d18b2ea9d None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:37.998839 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275900d-8ef6-4a24-af03-c72d18b2ea9d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:37.999142 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275900d-8ef6-4a24-af03-c72d18b2ea9d None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:37.999946 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d275900d-8ef6-4a24-af03-c72d18b2ea9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:38.000259 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d275900d-8ef6-4a24-af03-c72d18b2ea9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:38.001719 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d275900d-8ef6-4a24-af03-c72d18b2ea9d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:38.028317 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 305/610] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:29:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:38.034455 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d275900d-8ef6-4a24-af03-c72d18b2ea9d None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:38.085110 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7934c97-972c-449c-9566-266b77709625 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:38.115229 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:38.155396 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c4730-4f3e-4cc5-9ecd-36f7f50e2f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:38.156584 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c4730-4f3e-4cc5-9ecd-36f7f50e2f7d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:38.157102 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c4730-4f3e-4cc5-9ecd-36f7f50e2f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:38.157576 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c4730-4f3e-4cc5-9ecd-36f7f50e2f7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:38.186260 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d275900d-8ef6-4a24-af03-c72d18b2ea9d None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:38.187668 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c4730-4f3e-4cc5-9ecd-36f7f50e2f7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:38.189355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c4730-4f3e-4cc5-9ecd-36f7f50e2f7d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:38.222749 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112cd6a3-3a20-4d7f-9478-d59535008367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:38.223626 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112cd6a3-3a20-4d7f-9478-d59535008367 None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:38.223988 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112cd6a3-3a20-4d7f-9478-d59535008367 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:38.225223 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112cd6a3-3a20-4d7f-9478-d59535008367 None admin] PATH_INFO: `/v3/projects/d488c79cb1704e9496fc196d8aa6c104/users/b47cec76cb1c4afb8e011ff04a885f96/roles/13dcf68cca9c44888c0fd27a78db3dbe` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:38.233635 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-487c4730-4f3e-4cc5-9ecd-36f7f50e2f7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:38.233635 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:38.233635 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:38.233635 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:38.233635 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:38.233635 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:38.233635 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:38.233635 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:38.233635 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:38.233635 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:38.233635 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:38.233635 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:38.233635 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:38.233635 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:38.233635 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:38.234565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:38.234565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:38.234565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:38.234565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:38.234565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:38.234565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:38.234565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:38.234565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:38.234565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:38.234565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:38.234565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:38.234565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:38.234565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:38.234565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:38.234565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:38.234565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:38.234565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:38.235517 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:38.235517 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:38.235517 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:38.235517 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:38.241568 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112cd6a3-3a20-4d7f-9478-d59535008367 None admin] RBAC: Authorizing `identity:create_grant(project_id=d488c79cb1704e9496fc196d8aa6c104, user_id=b47cec76cb1c4afb8e011ff04a885f96, role_id=13dcf68cca9c44888c0fd27a78db3dbe)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:38.242673 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112cd6a3-3a20-4d7f-9478-d59535008367 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:38.243429 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 306/611] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:38.249476 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c4730-4f3e-4cc5-9ecd-36f7f50e2f7d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:38.266959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 306/612] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Jun 15 14:29:38 2022] PUT /identity/v3/projects/d488c79cb1704e9496fc196d8aa6c104/users/b47cec76cb1c4afb8e011ff04a885f96/roles/13dcf68cca9c44888c0fd27a78db3dbe => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:29:38.378689 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:38.404076 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d9232-8725-40f2-9ae7-1123b2499fad None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:38.404546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d9232-8725-40f2-9ae7-1123b2499fad None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:38.405020 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d9232-8725-40f2-9ae7-1123b2499fad None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:38.407845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 307/613] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:29:38.412779 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c4730-4f3e-4cc5-9ecd-36f7f50e2f7d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:38.416035 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd388f-1d2e-4da2-baf1-6a8325ff574d None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:38.416529 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd388f-1d2e-4da2-baf1-6a8325ff574d None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:38.417008 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd388f-1d2e-4da2-baf1-6a8325ff574d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:38.451919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-6cbd388f-1d2e-4da2-baf1-6a8325ff574d None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:38.453141 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6cbd388f-1d2e-4da2-baf1-6a8325ff574d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:38.474185 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02236d31-6c0a-404d-a835-06fc372f791f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:38.474860 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02236d31-6c0a-404d-a835-06fc372f791f None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:38.475181 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02236d31-6c0a-404d-a835-06fc372f791f None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:38.475461 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02236d31-6c0a-404d-a835-06fc372f791f None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:38.476206 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02236d31-6c0a-404d-a835-06fc372f791f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:38.476515 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02236d31-6c0a-404d-a835-06fc372f791f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:38.477921 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02236d31-6c0a-404d-a835-06fc372f791f None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:38.511256 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 307/614] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:38.526417 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02236d31-6c0a-404d-a835-06fc372f791f None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:38.568600 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 308/615] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:29:38.625999 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd388f-1d2e-4da2-baf1-6a8325ff574d None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:38.656576 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:38.695486 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02236d31-6c0a-404d-a835-06fc372f791f None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:38.807747 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b385ebe7-7778-4015-a735-07d6b97d91da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:38.809080 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385ebe7-7778-4015-a735-07d6b97d91da None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:38.809362 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385ebe7-7778-4015-a735-07d6b97d91da None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:38.809630 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385ebe7-7778-4015-a735-07d6b97d91da None admin] PATH_INFO: `/v3/users/swift` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:38.809963 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd388f-1d2e-4da2-baf1-6a8325ff574d None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:38.821469 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:38.831587 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b385ebe7-7778-4015-a735-07d6b97d91da None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:38.833037 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b385ebe7-7778-4015-a735-07d6b97d91da None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:38.845937 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b2aa1-b393-44ea-925f-24c6e3345eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:38.846742 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b2aa1-b393-44ea-925f-24c6e3345eba None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:38.847056 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b2aa1-b393-44ea-925f-24c6e3345eba None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:38.847340 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b2aa1-b393-44ea-925f-24c6e3345eba None admin] PATH_INFO: `/v3/roles/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:38.858134 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b2aa1-b393-44ea-925f-24c6e3345eba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:38.865845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-db6b2aa1-b393-44ea-925f-24c6e3345eba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:38.865845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:38.865845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:38.865845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:38.865845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:38.865845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:38.865845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:38.865845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:38.865845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:38.865845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:38.865845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:38.865845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:38.865845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:38.865845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:38.865845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:38.865845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:38.868023 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:38.868023 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:38.868023 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:38.868023 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:38.868023 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:38.868023 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:38.868023 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:38.868023 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:38.868023 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:38.868023 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:38.868023 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:38.868023 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:38.868023 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:38.868023 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:38.868023 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:38.868023 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:38.869937 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:38.869937 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:38.869937 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:38.869937 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:38.869937 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:38.869937 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:38.869937 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:38.869937 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:38.869937 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:38.869937 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:38.869937 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:38.869937 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:38.869937 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 309/616] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Jun 15 14:29:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:29:38.871942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-b385ebe7-7778-4015-a735-07d6b97d91da None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:29:38.871942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:38.871942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:38.871942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:38.871942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:38.871942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:38.871942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:38.871942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:38.871942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:38.871942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:38.871942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:38.871942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:38.871942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:29:38.871942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:29:38.871942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:29:38.871942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:29:38.872797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:38.872797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:38.872797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:29:38.872797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:38.872797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:29:38.872797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:38.872797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:38.872797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:38.872797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:38.872797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:38.872797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:38.872797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:38.872797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:38.872797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:38.872797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:38.872797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:38.872797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:38.872797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:38.873699 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:38.873699 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:38.873699 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:29:38.873699 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:29:38.873699 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:29:38.873699 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:29:38.873699 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:29:38.873699 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:29:38.873699 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:29:38.873699 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:38.873699 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 308/617] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:29:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:29:38.879012 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b2aa1-b393-44ea-925f-24c6e3345eba None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:38.880029 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b385ebe7-7778-4015-a735-07d6b97d91da None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:38.983364 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:39.003876 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b2aa1-b393-44ea-925f-24c6e3345eba None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:39.100905 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b385ebe7-7778-4015-a735-07d6b97d91da None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:39.105093 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f9b41-d5c5-45c2-ba78-375d853ed2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:39.105984 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f9b41-d5c5-45c2-ba78-375d853ed2a4 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:39.106465 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f9b41-d5c5-45c2-ba78-375d853ed2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:39.106891 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f9b41-d5c5-45c2-ba78-375d853ed2a4 None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:39.107905 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f9b41-d5c5-45c2-ba78-375d853ed2a4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:39.108388 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f9b41-d5c5-45c2-ba78-375d853ed2a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:39.109991 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f9b41-d5c5-45c2-ba78-375d853ed2a4 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:39.145684 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 310/618] 10.222.0.45 () {56 vars in 1135 bytes} [Wed Jun 15 14:29:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:39.152658 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f9b41-d5c5-45c2-ba78-375d853ed2a4 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:39.181786 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef964c5-6060-4c00-931b-b69a354e853c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:39.183050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef964c5-6060-4c00-931b-b69a354e853c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:39.183494 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef964c5-6060-4c00-931b-b69a354e853c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:39.183985 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef964c5-6060-4c00-931b-b69a354e853c None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:39.184902 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef964c5-6060-4c00-931b-b69a354e853c None admin] RBAC: Adding query filter params (name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:39.185361 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef964c5-6060-4c00-931b-b69a354e853c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:39.186923 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef964c5-6060-4c00-931b-b69a354e853c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:39.211069 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 309/619] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Jun 15 14:29:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:39.217184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef964c5-6060-4c00-931b-b69a354e853c None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:39.316865 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:39.335717 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f9b41-d5c5-45c2-ba78-375d853ed2a4 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:39.402684 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef964c5-6060-4c00-931b-b69a354e853c None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:39.412102 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9fbf78-fdc2-440c-877d-514746f7cb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:39.412907 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9fbf78-fdc2-440c-877d-514746f7cb64 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:39.413191 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9fbf78-fdc2-440c-877d-514746f7cb64 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:39.413470 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9fbf78-fdc2-440c-877d-514746f7cb64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:39.426513 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:39.431943 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9fbf78-fdc2-440c-877d-514746f7cb64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:39.433670 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9fbf78-fdc2-440c-877d-514746f7cb64 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:39.466019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-9a9fbf78-fdc2-440c-877d-514746f7cb64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:39.466019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:39.466019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:39.466019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:39.466019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:39.466019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:39.466019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:39.466019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:39.466019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:39.466019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:39.466019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:39.466019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:39.466019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:39.466019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:39.466019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:39.467042 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:39.467042 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:39.467042 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:39.467042 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:39.467042 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:39.467042 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:39.467042 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:39.467042 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:39.467042 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:39.467042 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:39.467042 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:39.467042 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:39.467042 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:39.467042 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:39.467042 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:39.467042 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:39.467042 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:39.468022 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:39.468022 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:39.468022 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:39.468022 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:39.472575 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 311/620] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:39.478633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9fbf78-fdc2-440c-877d-514746f7cb64 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:39.483156 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce22ced0-81ee-4986-b761-3d52f8b8c27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:39.484048 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22ced0-81ee-4986-b761-3d52f8b8c27e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:39.484339 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22ced0-81ee-4986-b761-3d52f8b8c27e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:39.484616 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22ced0-81ee-4986-b761-3d52f8b8c27e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:39.514007 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce22ced0-81ee-4986-b761-3d52f8b8c27e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:39.515712 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce22ced0-81ee-4986-b761-3d52f8b8c27e None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:39.552184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-ce22ced0-81ee-4986-b761-3d52f8b8c27e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:39.552184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:39.552184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:39.552184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:39.552184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:39.552184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:39.552184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:39.552184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:39.552184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:39.552184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:39.552184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:39.552184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:39.552184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:39.552184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:39.552184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:39.554355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:39.554355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:39.554355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:39.554355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:39.554355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:39.554355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:39.554355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:39.554355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:39.554355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:39.554355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:39.554355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:39.554355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:39.554355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:39.554355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:39.554355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:39.554355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:39.554355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:39.556441 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:39.556441 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:39.556441 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:39.556441 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:39.556441 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 310/621] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:29:39.562933 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce22ced0-81ee-4986-b761-3d52f8b8c27e None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:39.591537 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9fbf78-fdc2-440c-877d-514746f7cb64 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:39.630632 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d228c6-fd56-47d9-9192-100feb8c466f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:39.630632 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d228c6-fd56-47d9-9192-100feb8c466f None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:39.630632 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d228c6-fd56-47d9-9192-100feb8c466f None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:39.630632 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d228c6-fd56-47d9-9192-100feb8c466f None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:39.631372 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d228c6-fd56-47d9-9192-100feb8c466f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:39.631770 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d228c6-fd56-47d9-9192-100feb8c466f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:39.633205 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d228c6-fd56-47d9-9192-100feb8c466f None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:39.646266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 312/622] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:39.652453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d228c6-fd56-47d9-9192-100feb8c466f None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:39.713320 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce22ced0-81ee-4986-b761-3d52f8b8c27e None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:39.731653 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:39.732554 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:39.760853 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6895194-cd28-412e-b70b-8405130f9f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:39.762071 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6895194-cd28-412e-b70b-8405130f9f59 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:39.762358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6895194-cd28-412e-b70b-8405130f9f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:39.762637 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6895194-cd28-412e-b70b-8405130f9f59 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:39.763523 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6895194-cd28-412e-b70b-8405130f9f59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:39.763843 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6895194-cd28-412e-b70b-8405130f9f59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:39.765702 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6895194-cd28-412e-b70b-8405130f9f59 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:39.790251 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 311/623] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:39.796273 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6895194-cd28-412e-b70b-8405130f9f59 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:39.812093 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d228c6-fd56-47d9-9192-100feb8c466f None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:39.877783 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d66c6-9802-4a9f-bf69-9f21afed090c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:39.878502 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d66c6-9802-4a9f-bf69-9f21afed090c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:39.878790 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d66c6-9802-4a9f-bf69-9f21afed090c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:39.879089 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d66c6-9802-4a9f-bf69-9f21afed090c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:39.892488 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209d66c6-9802-4a9f-bf69-9f21afed090c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:39.897021 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209d66c6-9802-4a9f-bf69-9f21afed090c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:39.924032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-209d66c6-9802-4a9f-bf69-9f21afed090c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:39.924032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:39.924032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:39.924032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:39.924032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:39.924032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:39.924032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:39.924032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:39.924032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:39.924032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:39.924032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:39.924032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:39.924032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:29:39.924032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:29:39.924032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:29:39.924981 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:29:39.924981 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:39.924981 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:39.924981 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:39.924981 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:39.924981 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:39.924981 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:39.924981 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:39.924981 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:39.924981 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:39.924981 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:39.924981 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:39.924981 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:39.924981 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:39.924981 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:39.924981 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:39.924981 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:39.925835 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:29:39.925835 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:29:39.925835 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:29:39.925835 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:29:39.925835 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:29:39.925835 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:29:39.925835 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:39.925835 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:39.926724 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 313/624] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:29:39.933725 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d66c6-9802-4a9f-bf69-9f21afed090c None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:39.972564 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6895194-cd28-412e-b70b-8405130f9f59 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:40.009833 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0556241-7ae6-4324-a129-848a366aa4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:40.010657 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0556241-7ae6-4324-a129-848a366aa4df None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:40.010943 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0556241-7ae6-4324-a129-848a366aa4df None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:40.011970 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0556241-7ae6-4324-a129-848a366aa4df None admin] PATH_INFO: `/v3/users/nova` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:40.021513 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0556241-7ae6-4324-a129-848a366aa4df None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:40.023313 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0556241-7ae6-4324-a129-848a366aa4df None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:40.040433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-c0556241-7ae6-4324-a129-848a366aa4df None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:29:40.040433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:40.040433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:40.040433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:40.040433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:40.040433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:40.040433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:40.040433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:40.040433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:40.040433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:40.040433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:40.040433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:40.040433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:29:40.040433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:29:40.040433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:29:40.040433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:29:40.041360 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:40.041360 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:40.041360 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:29:40.041360 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:40.041360 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:29:40.041360 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:40.041360 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:40.041360 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:40.041360 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:40.041360 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:40.041360 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:40.041360 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:40.041360 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:40.041360 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:40.041360 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:40.041360 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:40.041360 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:40.041360 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:40.042261 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:40.042261 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:40.042261 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:29:40.042261 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:29:40.042261 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:29:40.042261 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:29:40.042261 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:29:40.042261 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:29:40.042261 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:29:40.042261 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:40.042261 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 312/625] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Jun 15 14:29:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:40.049738 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0556241-7ae6-4324-a129-848a366aa4df None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:40.117248 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:40.141459 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d66c6-9802-4a9f-bf69-9f21afed090c None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:40.162417 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:40.210863 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b5ddbc-1c06-4bf2-856c-30d2b6ff2029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:40.211582 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b5ddbc-1c06-4bf2-856c-30d2b6ff2029 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:40.211866 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b5ddbc-1c06-4bf2-856c-30d2b6ff2029 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:40.212142 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b5ddbc-1c06-4bf2-856c-30d2b6ff2029 None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:40.213031 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b5ddbc-1c06-4bf2-856c-30d2b6ff2029 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:40.213348 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b5ddbc-1c06-4bf2-856c-30d2b6ff2029 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:40.214967 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b5ddbc-1c06-4bf2-856c-30d2b6ff2029 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:40.241602 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 314/626] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:29:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:40.248705 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b5ddbc-1c06-4bf2-856c-30d2b6ff2029 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:40.254129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0556241-7ae6-4324-a129-848a366aa4df None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:40.292661 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d23b5-8382-48a8-8575-1d4bf8d8382a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:40.293373 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d23b5-8382-48a8-8575-1d4bf8d8382a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:40.293658 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d23b5-8382-48a8-8575-1d4bf8d8382a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:40.293938 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d23b5-8382-48a8-8575-1d4bf8d8382a None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:40.297969 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07d23b5-8382-48a8-8575-1d4bf8d8382a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:40.297969 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07d23b5-8382-48a8-8575-1d4bf8d8382a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:40.297969 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07d23b5-8382-48a8-8575-1d4bf8d8382a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:40.315756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 313/627] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Jun 15 14:29:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:40.317092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf34d8-84f5-4e60-a281-c1e004bf5514 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:40.317424 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf34d8-84f5-4e60-a281-c1e004bf5514 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:40.317709 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf34d8-84f5-4e60-a281-c1e004bf5514 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:40.319206 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 314/628] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:40.321635 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf34d8-84f5-4e60-a281-c1e004bf5514 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:40.387905 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b5ddbc-1c06-4bf2-856c-30d2b6ff2029 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:40.423634 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad90383-ae94-4b90-a8e0-b1b212f2382b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:40.424716 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad90383-ae94-4b90-a8e0-b1b212f2382b None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:40.425148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad90383-ae94-4b90-a8e0-b1b212f2382b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:40.425421 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad90383-ae94-4b90-a8e0-b1b212f2382b None admin] PATH_INFO: `/v3/projects/d488c79cb1704e9496fc196d8aa6c104/users/641a81b3140f47119b5c2323a97a6681/roles/13dcf68cca9c44888c0fd27a78db3dbe` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:40.427545 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:40.442445 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad90383-ae94-4b90-a8e0-b1b212f2382b None admin] RBAC: Authorizing `identity:create_grant(project_id=d488c79cb1704e9496fc196d8aa6c104, user_id=641a81b3140f47119b5c2323a97a6681, role_id=13dcf68cca9c44888c0fd27a78db3dbe)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:40.443922 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad90383-ae94-4b90-a8e0-b1b212f2382b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:40.444165 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf34d8-84f5-4e60-a281-c1e004bf5514 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:40.458389 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 315/629] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Jun 15 14:29:40 2022] PUT /identity/v3/projects/d488c79cb1704e9496fc196d8aa6c104/users/641a81b3140f47119b5c2323a97a6681/roles/13dcf68cca9c44888c0fd27a78db3dbe => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:29:40.460715 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab505396-80fe-4076-9727-0c7139dc02e2 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:40.461521 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab505396-80fe-4076-9727-0c7139dc02e2 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:40.462021 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab505396-80fe-4076-9727-0c7139dc02e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:40.482364 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f19cf-c00b-4e6e-bffe-b7eafb6513ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:40.483100 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f19cf-c00b-4e6e-bffe-b7eafb6513ae None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:40.483387 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f19cf-c00b-4e6e-bffe-b7eafb6513ae None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:40.483710 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f19cf-c00b-4e6e-bffe-b7eafb6513ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:40.487035 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-ab505396-80fe-4076-9727-0c7139dc02e2 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:40.489990 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ab505396-80fe-4076-9727-0c7139dc02e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:40.515374 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:40.577184 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 316/630] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:40.578681 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374c87f-e680-4343-bcba-2acd390a0cd9 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:40.578972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374c87f-e680-4343-bcba-2acd390a0cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:40.579319 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374c87f-e680-4343-bcba-2acd390a0cd9 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:40.581431 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00f19cf-c00b-4e6e-bffe-b7eafb6513ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:40.583151 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00f19cf-c00b-4e6e-bffe-b7eafb6513ae None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:40.584750 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 317/631] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:40.585960 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e982d-4158-4845-9737-4a4abb0cdf67 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:40.586248 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e982d-4158-4845-9737-4a4abb0cdf67 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:40.586524 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e982d-4158-4845-9737-4a4abb0cdf67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:40.604410 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-fc0e982d-4158-4845-9737-4a4abb0cdf67 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:40.606248 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-fc0e982d-4158-4845-9737-4a4abb0cdf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:40.683280 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 318/632] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:40.687016 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e982d-4158-4845-9737-4a4abb0cdf67 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:40.702155 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:40.725668 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 315/633] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 14:29:40.726612 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f19cf-c00b-4e6e-bffe-b7eafb6513ae None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:40.800560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:40.826529 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e982d-4158-4845-9737-4a4abb0cdf67 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:40.845073 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f19cf-c00b-4e6e-bffe-b7eafb6513ae None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:40.864587 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5648ef-de86-4d89-81de-9126726e6abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:40.865753 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5648ef-de86-4d89-81de-9126726e6abb None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:40.866242 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5648ef-de86-4d89-81de-9126726e6abb None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:40.866757 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5648ef-de86-4d89-81de-9126726e6abb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:40.881136 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5648ef-de86-4d89-81de-9126726e6abb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:40.882842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98c55e1-7d4c-4f92-b215-ad9cbe7dd7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:40.883680 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c55e1-7d4c-4f92-b215-ad9cbe7dd7a8 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:40.883963 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c55e1-7d4c-4f92-b215-ad9cbe7dd7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:40.884243 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c55e1-7d4c-4f92-b215-ad9cbe7dd7a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:40.891251 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-be5648ef-de86-4d89-81de-9126726e6abb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:40.891251 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:40.891251 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:40.891251 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:40.891251 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:40.891251 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:40.891251 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:40.891251 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:40.891251 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:40.891251 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:40.891251 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:40.891251 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:40.891251 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:40.891251 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:40.891251 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:40.891251 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:40.892152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:40.892152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:40.892152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:40.892152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:40.892152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:40.892152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:40.892152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:40.892152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:40.892152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:40.892152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:40.892152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:40.892152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:40.892152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:40.892152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:40.892152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:40.892152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:40.892959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:40.892959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:40.892959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:40.892959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:40.892959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:40.892959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:40.892959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:40.892959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:40.892959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:40.892959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:40.892959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:40.892959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:40.895797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98c55e1-7d4c-4f92-b215-ad9cbe7dd7a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:40.896227 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 319/634] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Jun 15 14:29:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:40.897267 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5648ef-de86-4d89-81de-9126726e6abb None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:40.903925 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-a98c55e1-7d4c-4f92-b215-ad9cbe7dd7a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:40.903925 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:40.903925 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:40.903925 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:40.903925 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:40.903925 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:40.903925 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:40.903925 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:40.903925 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:40.903925 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:40.903925 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:40.903925 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:40.903925 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:40.903925 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:40.903925 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:40.903925 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:40.904879 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:40.904879 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:40.904879 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:40.904879 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:40.904879 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:40.904879 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:40.904879 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:40.904879 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:40.904879 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:40.904879 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:40.904879 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:40.904879 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:40.904879 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:40.904879 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:40.904879 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:40.904879 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:40.905677 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:40.905677 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:40.905677 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:40.905677 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:40.905677 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:40.905677 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:40.905677 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:40.905677 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:40.905677 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:40.905677 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:40.905677 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:40.905677 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:40.905677 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 316/635] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Jun 15 14:29:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:40.906392 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98c55e1-7d4c-4f92-b215-ad9cbe7dd7a8 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:40.960408 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:41.014633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5648ef-de86-4d89-81de-9126726e6abb None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:41.017193 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98c55e1-7d4c-4f92-b215-ad9cbe7dd7a8 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:41.060779 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ba4eb-00ef-433d-95ef-23d7137ac387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:41.062540 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ba4eb-00ef-433d-95ef-23d7137ac387 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:41.063034 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ba4eb-00ef-433d-95ef-23d7137ac387 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:41.063482 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ba4eb-00ef-433d-95ef-23d7137ac387 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:41.066374 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ec3a9-c9c8-4df6-ade4-3e5149fb7e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:41.067463 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ec3a9-c9c8-4df6-ade4-3e5149fb7e12 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:41.067886 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ec3a9-c9c8-4df6-ade4-3e5149fb7e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:41.068233 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ec3a9-c9c8-4df6-ade4-3e5149fb7e12 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:41.069055 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ec3a9-c9c8-4df6-ade4-3e5149fb7e12 None admin] RBAC: Adding query filter params (name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:41.069431 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ec3a9-c9c8-4df6-ade4-3e5149fb7e12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:41.070876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ec3a9-c9c8-4df6-ade4-3e5149fb7e12 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:41.082510 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ba4eb-00ef-433d-95ef-23d7137ac387 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:41.084576 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ba4eb-00ef-433d-95ef-23d7137ac387 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:41.085286 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 317/636] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Jun 15 14:29:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:41.091481 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ec3a9-c9c8-4df6-ade4-3e5149fb7e12 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:41.099572 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:41.112159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-876ba4eb-00ef-433d-95ef-23d7137ac387 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:41.112159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:41.112159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:41.112159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:41.112159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:41.112159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:41.112159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:41.112159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:41.112159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:41.112159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:41.112159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:41.112159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:41.112159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:41.112159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:41.112159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:41.113139 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:41.113139 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:41.113139 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:41.113139 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:41.113139 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:41.113139 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:41.113139 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:41.113139 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:41.113139 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:41.113139 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:41.113139 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:41.113139 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:41.113139 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:41.113139 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:41.113139 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:41.113139 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:41.113139 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:41.114001 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:41.114001 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:41.114001 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:41.114001 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:41.121283 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 320/637] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:41.121283 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ba4eb-00ef-433d-95ef-23d7137ac387 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:41.214313 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ec3a9-c9c8-4df6-ade4-3e5149fb7e12 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:41.223452 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:41.229735 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ba4eb-00ef-433d-95ef-23d7137ac387 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:41.251723 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b8a9a-066b-47d0-afe6-393f834684a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:41.252615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b8a9a-066b-47d0-afe6-393f834684a2 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:41.253105 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b8a9a-066b-47d0-afe6-393f834684a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:41.253648 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b8a9a-066b-47d0-afe6-393f834684a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:41.254395 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848b8a9a-066b-47d0-afe6-393f834684a2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:41.254704 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848b8a9a-066b-47d0-afe6-393f834684a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:41.256208 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848b8a9a-066b-47d0-afe6-393f834684a2 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:41.264345 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7b4c7-a0f4-4851-970f-0dfebe2e5349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:41.269635 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 318/638] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Jun 15 14:29:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:41.269791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7b4c7-a0f4-4851-970f-0dfebe2e5349 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:41.269791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7b4c7-a0f4-4851-970f-0dfebe2e5349 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:41.269791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7b4c7-a0f4-4851-970f-0dfebe2e5349 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:41.270129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b8a9a-066b-47d0-afe6-393f834684a2 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:41.279863 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd7b4c7-a0f4-4851-970f-0dfebe2e5349 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:41.281334 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd7b4c7-a0f4-4851-970f-0dfebe2e5349 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:41.299595 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-5cd7b4c7-a0f4-4851-970f-0dfebe2e5349 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:41.299595 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:41.299595 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:41.299595 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:41.299595 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:41.299595 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:41.299595 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:41.299595 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:41.299595 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:41.299595 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:41.299595 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:41.299595 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:41.299595 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:41.299595 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:41.299595 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:41.300496 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:41.300496 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:41.300496 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:41.300496 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:41.300496 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:41.300496 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:41.300496 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:41.300496 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:41.300496 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:41.300496 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:41.300496 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:41.300496 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:41.300496 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:41.300496 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:41.300496 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:41.300496 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:41.300496 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:41.301377 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:41.301377 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:41.301377 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:41.301377 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:41.301377 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 321/639] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:41.302051 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a5dcd-7e98-4a73-b51a-79bcfcc078e8 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:41.302283 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a5dcd-7e98-4a73-b51a-79bcfcc078e8 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:41.302562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a5dcd-7e98-4a73-b51a-79bcfcc078e8 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:41.304029 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 322/640] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:41.309039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a5dcd-7e98-4a73-b51a-79bcfcc078e8 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:41.364727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:41.381880 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b8a9a-066b-47d0-afe6-393f834684a2 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:41.420071 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19437fcf-5de2-4aa8-959b-2189e3504c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:41.420732 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19437fcf-5de2-4aa8-959b-2189e3504c48 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:41.421023 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19437fcf-5de2-4aa8-959b-2189e3504c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:41.421280 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19437fcf-5de2-4aa8-959b-2189e3504c48 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:41.422025 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19437fcf-5de2-4aa8-959b-2189e3504c48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:41.422335 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19437fcf-5de2-4aa8-959b-2189e3504c48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:41.423732 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19437fcf-5de2-4aa8-959b-2189e3504c48 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:41.425642 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a5dcd-7e98-4a73-b51a-79bcfcc078e8 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:41.436578 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 319/641] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:41.437987 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19437fcf-5de2-4aa8-959b-2189e3504c48 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:41.464916 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b43957-6181-4039-84be-6eaed957e4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:41.465666 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b43957-6181-4039-84be-6eaed957e4b9 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:41.465952 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b43957-6181-4039-84be-6eaed957e4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:41.466254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b43957-6181-4039-84be-6eaed957e4b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:41.474089 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:41.478035 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b43957-6181-4039-84be-6eaed957e4b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:41.479633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b43957-6181-4039-84be-6eaed957e4b9 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:41.497801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-a0b43957-6181-4039-84be-6eaed957e4b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:41.497801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:41.497801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:41.497801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:41.497801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:41.497801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:41.497801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:41.497801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:41.497801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:41.497801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:41.497801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:41.497801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:41.497801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:41.497801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:41.497801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:41.498944 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:41.498944 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:41.498944 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:41.498944 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:41.498944 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:41.498944 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:41.498944 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:41.498944 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:41.498944 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:41.498944 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:41.498944 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:41.498944 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:41.498944 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:41.498944 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:41.498944 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:41.498944 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:41.498944 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:41.500084 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:41.500084 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:41.500084 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:41.500084 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:41.505507 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 323/642] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:41.505507 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62a37d-0410-4ffe-bc64-f400dd5878dd None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:41.505507 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62a37d-0410-4ffe-bc64-f400dd5878dd None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:41.505507 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62a37d-0410-4ffe-bc64-f400dd5878dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:41.519226 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-2c62a37d-0410-4ffe-bc64-f400dd5878dd None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:41.520754 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-2c62a37d-0410-4ffe-bc64-f400dd5878dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:41.557811 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19437fcf-5de2-4aa8-959b-2189e3504c48 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:41.586935 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 324/643] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:41.587857 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c62a37d-0410-4ffe-bc64-f400dd5878dd None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:41.606524 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804ca1f-08da-4095-8d00-7947941bc81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:41.607810 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804ca1f-08da-4095-8d00-7947941bc81c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:41.608764 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804ca1f-08da-4095-8d00-7947941bc81c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:41.609248 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804ca1f-08da-4095-8d00-7947941bc81c None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:41.610236 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8804ca1f-08da-4095-8d00-7947941bc81c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:41.610646 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8804ca1f-08da-4095-8d00-7947941bc81c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:41.612604 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8804ca1f-08da-4095-8d00-7947941bc81c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:41.618502 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:41.628742 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 320/644] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:41.628742 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804ca1f-08da-4095-8d00-7947941bc81c None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:41.700298 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c62a37d-0410-4ffe-bc64-f400dd5878dd None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:41.729092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:41.741152 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804ca1f-08da-4095-8d00-7947941bc81c None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:41.742947 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0d2c2-b0db-4e11-989c-a579b84d3a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:41.744141 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0d2c2-b0db-4e11-989c-a579b84d3a3d None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:41.744141 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0d2c2-b0db-4e11-989c-a579b84d3a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:41.744373 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0d2c2-b0db-4e11-989c-a579b84d3a3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:41.751419 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef0d2c2-b0db-4e11-989c-a579b84d3a3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:41.752883 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef0d2c2-b0db-4e11-989c-a579b84d3a3d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:41.766652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-2ef0d2c2-b0db-4e11-989c-a579b84d3a3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:41.766652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:41.766652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:41.766652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:41.766652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:41.766652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:41.766652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:41.766652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:41.766652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:41.766652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:41.766652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:41.766652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:41.766652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:29:41.766652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:29:41.766652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:29:41.767723 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:29:41.767723 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:41.767723 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:41.767723 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:41.767723 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:41.767723 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:41.767723 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:41.767723 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:41.767723 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:41.767723 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:41.767723 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:41.767723 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:41.767723 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:41.767723 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:41.767723 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:41.767723 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:41.767723 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:41.768787 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:29:41.768787 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:29:41.768787 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:29:41.768787 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:29:41.768787 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:29:41.768787 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:29:41.768787 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:41.768787 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:41.768787 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 325/645] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:41.769529 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0d2c2-b0db-4e11-989c-a579b84d3a3d None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:41.783453 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f839a-70f4-49bb-b44c-d1fa343b38d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:41.784219 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f839a-70f4-49bb-b44c-d1fa343b38d5 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:41.784510 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f839a-70f4-49bb-b44c-d1fa343b38d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:41.784787 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f839a-70f4-49bb-b44c-d1fa343b38d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:41.785516 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f839a-70f4-49bb-b44c-d1fa343b38d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:41.785828 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f839a-70f4-49bb-b44c-d1fa343b38d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:41.787236 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f839a-70f4-49bb-b44c-d1fa343b38d5 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:41.801130 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 321/646] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:41.801932 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f839a-70f4-49bb-b44c-d1fa343b38d5 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:41.866363 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:41.882067 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0d2c2-b0db-4e11-989c-a579b84d3a3d None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:41.907123 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f839a-70f4-49bb-b44c-d1fa343b38d5 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:41.918038 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce78a4d-7cea-4b94-8510-4c08b59fbebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:41.918831 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce78a4d-7cea-4b94-8510-4c08b59fbebd None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:41.919157 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce78a4d-7cea-4b94-8510-4c08b59fbebd None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:41.919432 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce78a4d-7cea-4b94-8510-4c08b59fbebd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:41.929426 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce78a4d-7cea-4b94-8510-4c08b59fbebd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:41.937109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-8ce78a4d-7cea-4b94-8510-4c08b59fbebd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:41.937109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:41.937109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:41.937109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:41.937109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:41.937109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:41.937109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:41.937109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:41.937109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:41.937109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:41.937109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:41.937109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:41.937109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:41.937109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:41.937109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:41.937109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:41.938032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:41.938032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:41.938032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:41.938032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:41.938032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:41.938032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:41.938032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:41.938032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:41.938032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:41.938032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:41.938032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:41.938032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:41.938032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:41.938032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:41.938032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:41.938032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:41.938869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:41.938869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:41.938869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:41.938869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:41.938869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:41.938869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:41.938869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:41.938869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:41.938869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:41.938869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:41.938869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:41.938869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:41.938869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 326/647] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Jun 15 14:29:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:41.939594 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce78a4d-7cea-4b94-8510-4c08b59fbebd None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:41.950837 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ddcf9-aebd-4008-b425-98d06e21d4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:41.951980 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ddcf9-aebd-4008-b425-98d06e21d4c1 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:41.951980 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ddcf9-aebd-4008-b425-98d06e21d4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:41.952261 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ddcf9-aebd-4008-b425-98d06e21d4c1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:41.959670 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ddcf9-aebd-4008-b425-98d06e21d4c1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:41.961116 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ddcf9-aebd-4008-b425-98d06e21d4c1 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:41.976715 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-cf6ddcf9-aebd-4008-b425-98d06e21d4c1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 14:29:41.976715 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:41.976715 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:41.976715 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:41.976715 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:41.976715 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:41.976715 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:41.976715 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:41.976715 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:41.976715 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:41.976715 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:41.976715 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:41.976715 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:29:41.976715 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:29:41.976715 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:29:41.976715 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:29:41.977842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:41.977842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:41.977842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:29:41.977842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:41.977842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:29:41.977842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:41.977842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:41.977842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:41.977842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:41.977842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:41.977842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:41.977842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:41.977842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:41.977842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:41.977842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:41.977842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:41.977842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:41.977842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:41.978869 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:41.978869 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:41.978869 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:29:41.978869 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:29:41.978869 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:29:41.978869 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:29:41.978869 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:29:41.978869 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:29:41.978869 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 14:29:41.978869 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:41.982678 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 322/648] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Jun 15 14:29:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:41.983501 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ddcf9-aebd-4008-b425-98d06e21d4c1 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:41.995485 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:42.054330 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce78a4d-7cea-4b94-8510-4c08b59fbebd None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:42.086466 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ddcf9-aebd-4008-b425-98d06e21d4c1 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:42.091282 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1515e-ff41-4cb6-a3f4-82910ffa58e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:42.092063 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1515e-ff41-4cb6-a3f4-82910ffa58e2 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:42.092346 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1515e-ff41-4cb6-a3f4-82910ffa58e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:42.092621 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1515e-ff41-4cb6-a3f4-82910ffa58e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:42.093810 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1515e-ff41-4cb6-a3f4-82910ffa58e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:42.094126 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1515e-ff41-4cb6-a3f4-82910ffa58e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:42.096073 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1515e-ff41-4cb6-a3f4-82910ffa58e2 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:42.110847 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 327/649] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:29:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:42.111783 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1515e-ff41-4cb6-a3f4-82910ffa58e2 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:42.123934 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:42.129496 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95bc95a-1484-4e01-83e2-b147f4e10a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:42.130307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bc95a-1484-4e01-83e2-b147f4e10a0d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:42.130592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bc95a-1484-4e01-83e2-b147f4e10a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:42.130958 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bc95a-1484-4e01-83e2-b147f4e10a0d None admin] PATH_INFO: `/v3/users/glance` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:42.138743 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95bc95a-1484-4e01-83e2-b147f4e10a0d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:42.140220 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95bc95a-1484-4e01-83e2-b147f4e10a0d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:42.155406 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-f95bc95a-1484-4e01-83e2-b147f4e10a0d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 15 14:29:42.155406 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:42.155406 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:42.155406 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:42.155406 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:42.155406 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:42.155406 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:42.155406 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:42.155406 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:42.155406 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:42.155406 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:42.155406 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:42.155406 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:29:42.155406 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:29:42.155406 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:29:42.155406 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:29:42.156318 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:42.156318 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:42.156318 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:29:42.156318 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:42.156318 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:29:42.156318 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:42.156318 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:42.156318 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:42.156318 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:42.156318 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:42.156318 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:42.156318 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:42.156318 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:42.156318 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:42.156318 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:42.156318 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:42.156318 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:42.156318 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:42.157232 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:42.157232 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:42.157232 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:29:42.157232 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:29:42.157232 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:29:42.157232 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:29:42.157232 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:29:42.157232 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:29:42.157232 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 15 14:29:42.157232 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:42.157232 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 323/650] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:42.158710 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95bc95a-1484-4e01-83e2-b147f4e10a0d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:42.226803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1515e-ff41-4cb6-a3f4-82910ffa58e2 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:42.231175 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:42.263553 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95bc95a-1484-4e01-83e2-b147f4e10a0d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:42.267612 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8060c2a1-b0f7-4b52-8c5a-a0bdd09f751a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:42.268362 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8060c2a1-b0f7-4b52-8c5a-a0bdd09f751a None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:42.268722 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8060c2a1-b0f7-4b52-8c5a-a0bdd09f751a None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:42.269071 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8060c2a1-b0f7-4b52-8c5a-a0bdd09f751a None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:42.270161 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8060c2a1-b0f7-4b52-8c5a-a0bdd09f751a None admin] RBAC: Adding query filter params (name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:42.270548 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8060c2a1-b0f7-4b52-8c5a-a0bdd09f751a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:42.272061 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8060c2a1-b0f7-4b52-8c5a-a0bdd09f751a None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:42.281230 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 328/651] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Jun 15 14:29:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:42.282041 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8060c2a1-b0f7-4b52-8c5a-a0bdd09f751a None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:42.305301 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27a06e9-eac9-4755-bfb2-b9f84f73fcc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:42.305977 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a06e9-eac9-4755-bfb2-b9f84f73fcc8 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:42.306260 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a06e9-eac9-4755-bfb2-b9f84f73fcc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:42.306532 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a06e9-eac9-4755-bfb2-b9f84f73fcc8 None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:42.307396 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27a06e9-eac9-4755-bfb2-b9f84f73fcc8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:42.307707 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27a06e9-eac9-4755-bfb2-b9f84f73fcc8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:42.309168 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27a06e9-eac9-4755-bfb2-b9f84f73fcc8 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:42.327166 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 324/652] 10.222.0.45 () {56 vars in 1139 bytes} [Wed Jun 15 14:29:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:42.332573 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27a06e9-eac9-4755-bfb2-b9f84f73fcc8 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:42.357423 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:42.392207 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8060c2a1-b0f7-4b52-8c5a-a0bdd09f751a None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:42.430926 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f52879-f1d3-4ae3-bf27-f4a4451bc1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:42.431607 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f52879-f1d3-4ae3-bf27-f4a4451bc1d7 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:42.431895 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f52879-f1d3-4ae3-bf27-f4a4451bc1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:42.432195 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f52879-f1d3-4ae3-bf27-f4a4451bc1d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:42.433317 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f52879-f1d3-4ae3-bf27-f4a4451bc1d7 None admin] RBAC: Adding query filter params (role.id=13dcf68cca9c44888c0fd27a78db3dbe, scope.project.id=d488c79cb1704e9496fc196d8aa6c104, user.id=f3185bd2458c4e22830975700888e9be) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:42.433682 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f52879-f1d3-4ae3-bf27-f4a4451bc1d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:42.435411 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f52879-f1d3-4ae3-bf27-f4a4451bc1d7 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:42.442038 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 329/653] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:29:42 2022] GET /identity/v3/role_assignments?user.id=f3185bd2458c4e22830975700888e9be&scope.project.id=d488c79cb1704e9496fc196d8aa6c104&role.id=13dcf68cca9c44888c0fd27a78db3dbe => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:42.443354 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27a06e9-eac9-4755-bfb2-b9f84f73fcc8 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:42.452146 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f52879-f1d3-4ae3-bf27-f4a4451bc1d7 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:42.485093 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:42.497400 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed14f2-0878-4b77-bb9b-2ae374b083c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:42.498350 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed14f2-0878-4b77-bb9b-2ae374b083c4 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:42.499133 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed14f2-0878-4b77-bb9b-2ae374b083c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:42.499556 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed14f2-0878-4b77-bb9b-2ae374b083c4 None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:42.501659 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ed14f2-0878-4b77-bb9b-2ae374b083c4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:42.502175 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ed14f2-0878-4b77-bb9b-2ae374b083c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:42.503855 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ed14f2-0878-4b77-bb9b-2ae374b083c4 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:42.533027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 325/654] 10.222.0.45 () {56 vars in 1137 bytes} [Wed Jun 15 14:29:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:42.534197 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed14f2-0878-4b77-bb9b-2ae374b083c4 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:42.599064 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f52879-f1d3-4ae3-bf27-f4a4451bc1d7 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:42.650277 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a773623b-bcb2-4d48-8b06-20e040fec485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:42.650953 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed14f2-0878-4b77-bb9b-2ae374b083c4 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:42.651418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773623b-bcb2-4d48-8b06-20e040fec485 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:42.651704 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773623b-bcb2-4d48-8b06-20e040fec485 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:42.652013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773623b-bcb2-4d48-8b06-20e040fec485 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:42.655274 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:42.670442 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a773623b-bcb2-4d48-8b06-20e040fec485 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:42.671917 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a773623b-bcb2-4d48-8b06-20e040fec485 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:42.692418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-a773623b-bcb2-4d48-8b06-20e040fec485 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:42.692418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:42.692418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:42.692418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:42.692418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:42.692418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:42.692418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:42.692418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:42.692418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:42.692418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:42.692418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:42.692418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:42.692418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:42.692418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:42.692418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:42.693532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:42.693532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:42.693532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:42.693532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:42.693532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:42.693532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:42.693532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:42.693532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:42.693532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:42.693532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:42.693532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:42.693532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:42.693532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:42.693532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:42.693532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:42.693532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:42.693532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:42.695184 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:42.695184 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:42.695184 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:42.695184 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:42.695184 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 330/655] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:42.695184 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a773623b-bcb2-4d48-8b06-20e040fec485 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:42.703050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab2ed2-36dd-4ef4-9c97-4c63e6644a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:42.703466 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab2ed2-36dd-4ef4-9c97-4c63e6644a4b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:42.703776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab2ed2-36dd-4ef4-9c97-4c63e6644a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:42.704148 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab2ed2-36dd-4ef4-9c97-4c63e6644a4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:42.768282 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:42.783213 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab2ed2-36dd-4ef4-9c97-4c63e6644a4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:42.784711 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab2ed2-36dd-4ef4-9c97-4c63e6644a4b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:42.826502 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a773623b-bcb2-4d48-8b06-20e040fec485 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:42.861952 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417da8aa-8186-4852-a8eb-3ef231cf7707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:42.862739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417da8aa-8186-4852-a8eb-3ef231cf7707 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:42.863159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417da8aa-8186-4852-a8eb-3ef231cf7707 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:42.863528 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417da8aa-8186-4852-a8eb-3ef231cf7707 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:42.891184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 326/656] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:42.892038 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab2ed2-36dd-4ef4-9c97-4c63e6644a4b None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:42.925552 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:42.940625 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417da8aa-8186-4852-a8eb-3ef231cf7707 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:42.942072 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417da8aa-8186-4852-a8eb-3ef231cf7707 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:42.986041 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:43.003237 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab2ed2-36dd-4ef4-9c97-4c63e6644a4b None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:43.041129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6fac69-c9c4-4ce3-9036-6936e2e1b1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:43.041720 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fac69-c9c4-4ce3-9036-6936e2e1b1f1 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:43.042005 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fac69-c9c4-4ce3-9036-6936e2e1b1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:43.042521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fac69-c9c4-4ce3-9036-6936e2e1b1f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:43.043026 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6fac69-c9c4-4ce3-9036-6936e2e1b1f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:43.043384 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6fac69-c9c4-4ce3-9036-6936e2e1b1f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:43.044779 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6fac69-c9c4-4ce3-9036-6936e2e1b1f1 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:43.046391 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 331/657] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:43.047652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f3157-a531-4006-b1af-f3b0e1b2337c None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:43.047946 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f3157-a531-4006-b1af-f3b0e1b2337c None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:43.048225 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f3157-a531-4006-b1af-f3b0e1b2337c None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:43.049747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 332/658] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:43.050466 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f3157-a531-4006-b1af-f3b0e1b2337c None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:43.060609 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 327/659] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:43.061783 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6fac69-c9c4-4ce3-9036-6936e2e1b1f1 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:43.143092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:43.159215 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f3157-a531-4006-b1af-f3b0e1b2337c None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:43.179437 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6fac69-c9c4-4ce3-9036-6936e2e1b1f1 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:43.192947 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9054368e-882a-4fcc-ae2c-3d2ec38a2a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:43.193731 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054368e-882a-4fcc-ae2c-3d2ec38a2a31 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:43.194011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054368e-882a-4fcc-ae2c-3d2ec38a2a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:43.194282 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054368e-882a-4fcc-ae2c-3d2ec38a2a31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:43.205086 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9054368e-882a-4fcc-ae2c-3d2ec38a2a31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:43.206527 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9054368e-882a-4fcc-ae2c-3d2ec38a2a31 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:43.216179 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a10e982-ffd7-48ba-903f-2a62ee46cb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:43.216981 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10e982-ffd7-48ba-903f-2a62ee46cb86 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:43.217267 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10e982-ffd7-48ba-903f-2a62ee46cb86 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:43.217548 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10e982-ffd7-48ba-903f-2a62ee46cb86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:43.223682 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-9054368e-882a-4fcc-ae2c-3d2ec38a2a31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:43.223682 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:43.223682 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:43.223682 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:43.223682 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:43.223682 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:43.223682 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:43.223682 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:43.223682 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:43.223682 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:43.223682 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:43.223682 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:43.223682 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:43.223682 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:43.223682 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:43.224532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:43.224532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:43.224532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:43.224532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:43.224532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:43.224532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:43.224532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:43.224532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:43.224532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:43.224532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:43.224532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:43.224532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:43.224532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:43.224532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:43.224532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:43.224532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:43.224532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:43.225436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:43.225436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:43.225436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:43.225436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:43.225436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 333/660] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:43.226115 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5e7ca-57c6-4a7e-891a-4301b13a9dcb None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:43.226341 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5e7ca-57c6-4a7e-891a-4301b13a9dcb None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:43.226615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5e7ca-57c6-4a7e-891a-4301b13a9dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:43.233983 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a10e982-ffd7-48ba-903f-2a62ee46cb86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:43.235532 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a10e982-ffd7-48ba-903f-2a62ee46cb86 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:43.242286 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:43.242563 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-36a5e7ca-57c6-4a7e-891a-4301b13a9dcb None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:43.243758 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-36a5e7ca-57c6-4a7e-891a-4301b13a9dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:43.255452 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-3a10e982-ffd7-48ba-903f-2a62ee46cb86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:43.255452 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:43.255452 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:43.255452 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:43.255452 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:43.255452 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:43.255452 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:43.255452 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:43.255452 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:43.255452 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:43.255452 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:43.255452 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:43.255452 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:43.255452 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:43.255452 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:43.256345 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:43.256345 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:43.256345 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:43.256345 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:43.256345 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:43.256345 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:43.256345 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:43.256345 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:43.256345 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:43.256345 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:43.256345 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:43.256345 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:43.256345 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:43.256345 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:43.256345 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:43.256345 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:43.256345 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:43.257214 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:43.257214 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:43.257214 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:43.257214 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:43.257214 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 328/661] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:43.257897 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a10e982-ffd7-48ba-903f-2a62ee46cb86 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:43.320811 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 334/662] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:43.321819 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a5e7ca-57c6-4a7e-891a-4301b13a9dcb None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:43.376861 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a10e982-ffd7-48ba-903f-2a62ee46cb86 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:43.399950 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:43.415469 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e0c47-5e62-4f54-b3d1-b77960a07309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:43.416272 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e0c47-5e62-4f54-b3d1-b77960a07309 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:43.416694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e0c47-5e62-4f54-b3d1-b77960a07309 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:43.416894 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e0c47-5e62-4f54-b3d1-b77960a07309 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:43.426264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8e0c47-5e62-4f54-b3d1-b77960a07309 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:43.427741 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8e0c47-5e62-4f54-b3d1-b77960a07309 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:43.439700 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a5e7ca-57c6-4a7e-891a-4301b13a9dcb None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:43.445226 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-0f8e0c47-5e62-4f54-b3d1-b77960a07309 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 14:29:43.445226 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:43.445226 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:43.445226 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:43.445226 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:43.445226 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:43.445226 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:43.445226 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:43.445226 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:43.445226 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:43.445226 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:43.445226 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:43.445226 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:29:43.445226 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:29:43.445226 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:29:43.445226 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:29:43.446199 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:43.446199 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:43.446199 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:29:43.446199 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:43.446199 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:29:43.446199 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:43.446199 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:43.446199 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:43.446199 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:43.446199 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:43.446199 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:43.446199 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:43.446199 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:43.446199 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:43.446199 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:43.446199 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:43.446199 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:43.446199 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:43.447265 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:43.447265 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:43.447265 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:29:43.447265 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:29:43.447265 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:29:43.447265 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:29:43.447265 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:29:43.447265 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:29:43.447265 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 14:29:43.447265 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:43.447265 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 329/663] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:43.447920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e0c47-5e62-4f54-b3d1-b77960a07309 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:43.479145 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe92f9-27a4-4a47-8486-ac70f41804b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:43.479817 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe92f9-27a4-4a47-8486-ac70f41804b6 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:43.480699 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe92f9-27a4-4a47-8486-ac70f41804b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:43.480982 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe92f9-27a4-4a47-8486-ac70f41804b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:43.481722 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe92f9-27a4-4a47-8486-ac70f41804b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:43.482015 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe92f9-27a4-4a47-8486-ac70f41804b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:43.483410 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe92f9-27a4-4a47-8486-ac70f41804b6 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:43.494407 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 335/664] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:43.495243 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe92f9-27a4-4a47-8486-ac70f41804b6 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:43.522735 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:43.568698 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e0c47-5e62-4f54-b3d1-b77960a07309 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:43.603703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f7b2e-2ab8-4035-8dd8-7ecc393d5151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:43.604293 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f7b2e-2ab8-4035-8dd8-7ecc393d5151 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:43.604580 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f7b2e-2ab8-4035-8dd8-7ecc393d5151 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:43.604859 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f7b2e-2ab8-4035-8dd8-7ecc393d5151 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:43.605674 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe92f9-27a4-4a47-8486-ac70f41804b6 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:43.606156 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026f7b2e-2ab8-4035-8dd8-7ecc393d5151 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:43.606465 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026f7b2e-2ab8-4035-8dd8-7ecc393d5151 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:43.607910 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026f7b2e-2ab8-4035-8dd8-7ecc393d5151 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:43.619688 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 330/665] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:43.620487 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f7b2e-2ab8-4035-8dd8-7ecc393d5151 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:43.638377 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:43.648350 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4074117-533f-46df-818d-7a918cadeb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:43.649170 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4074117-533f-46df-818d-7a918cadeb96 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:43.649453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4074117-533f-46df-818d-7a918cadeb96 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:43.649729 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4074117-533f-46df-818d-7a918cadeb96 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:43.661803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4074117-533f-46df-818d-7a918cadeb96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:43.670463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-a4074117-533f-46df-818d-7a918cadeb96 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:43.670463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:43.670463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:43.670463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:43.670463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:43.670463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:43.670463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:43.670463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:43.670463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:43.670463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:43.670463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:43.670463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:43.670463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:43.670463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:43.670463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:43.670463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:43.671438 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:43.671438 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:43.671438 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:43.671438 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:43.671438 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:43.671438 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:43.671438 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:43.671438 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:43.671438 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:43.671438 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:43.671438 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:43.671438 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:43.671438 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:43.671438 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:43.671438 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:43.671438 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:43.672306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:43.672306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:43.672306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:43.672306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:43.672306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:43.672306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:43.672306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:43.672306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:43.672306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:43.672306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:43.672306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:43.672306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:43.672306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 336/666] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Jun 15 14:29:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:43.672998 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4074117-533f-46df-818d-7a918cadeb96 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:43.746934 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f7b2e-2ab8-4035-8dd8-7ecc393d5151 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:43.765801 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:43.784743 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fdc998-e97f-45b0-8ca7-d114e28c7891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:43.785418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fdc998-e97f-45b0-8ca7-d114e28c7891 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:43.785708 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fdc998-e97f-45b0-8ca7-d114e28c7891 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:43.785990 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fdc998-e97f-45b0-8ca7-d114e28c7891 None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:43.786823 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fdc998-e97f-45b0-8ca7-d114e28c7891 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:43.787166 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fdc998-e97f-45b0-8ca7-d114e28c7891 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:43.788604 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fdc998-e97f-45b0-8ca7-d114e28c7891 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:43.792477 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4074117-533f-46df-818d-7a918cadeb96 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:43.806423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 331/667] 10.222.0.45 () {56 vars in 1137 bytes} [Wed Jun 15 14:29:43 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:43.807557 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fdc998-e97f-45b0-8ca7-d114e28c7891 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:43.833779 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95696c4-655f-4f0d-9463-a8549e77d431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:43.834959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95696c4-655f-4f0d-9463-a8549e77d431 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:43.835466 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95696c4-655f-4f0d-9463-a8549e77d431 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:43.835971 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95696c4-655f-4f0d-9463-a8549e77d431 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:43.843061 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95696c4-655f-4f0d-9463-a8549e77d431 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:43.844711 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95696c4-655f-4f0d-9463-a8549e77d431 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:43.858862 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-a95696c4-655f-4f0d-9463-a8549e77d431 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:43.858862 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:43.858862 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:43.858862 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:43.858862 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:43.858862 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:43.858862 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:43.858862 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:43.858862 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:43.858862 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:43.858862 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:43.858862 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:43.858862 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:29:43.858862 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:29:43.858862 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:29:43.859739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:29:43.859739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:43.859739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:43.859739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:43.859739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:43.859739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:43.859739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:43.859739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:43.859739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:43.859739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:43.859739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:43.859739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:43.859739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:43.859739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:43.859739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:43.859739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:43.859739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:43.865756 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:29:43.865756 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:29:43.865756 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:29:43.865756 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:29:43.865756 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:29:43.865756 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:29:43.865756 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:43.865756 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:43.865756 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 337/668] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:43.865756 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95696c4-655f-4f0d-9463-a8549e77d431 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:43.916704 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:43.918153 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fdc998-e97f-45b0-8ca7-d114e28c7891 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:43.954175 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed6a84-72b0-4981-85aa-7cdf03d59868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:43.955234 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed6a84-72b0-4981-85aa-7cdf03d59868 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:43.955715 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed6a84-72b0-4981-85aa-7cdf03d59868 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:43.956161 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed6a84-72b0-4981-85aa-7cdf03d59868 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:43.963391 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ed6a84-72b0-4981-85aa-7cdf03d59868 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:43.965060 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ed6a84-72b0-4981-85aa-7cdf03d59868 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:43.970427 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95696c4-655f-4f0d-9463-a8549e77d431 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:43.980096 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-87ed6a84-72b0-4981-85aa-7cdf03d59868 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:43.980096 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:43.980096 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:43.980096 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:43.980096 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:43.980096 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:43.980096 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:43.980096 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:43.980096 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:43.980096 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:43.980096 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:43.980096 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:43.980096 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:29:43.980096 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:29:43.980096 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:29:43.981028 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:29:43.981028 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:43.981028 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:43.981028 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:43.981028 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:43.981028 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:43.981028 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:43.981028 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:43.981028 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:43.981028 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:43.981028 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:43.981028 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:43.981028 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:43.981028 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:43.981028 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:43.981028 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:43.981028 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:43.981916 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:29:43.981916 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:29:43.981916 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:29:43.981916 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:29:43.981916 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:29:43.981916 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:29:43.981916 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:43.981916 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:43.983783 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 332/669] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:43.984735 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed6a84-72b0-4981-85aa-7cdf03d59868 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:44.012063 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d633f2bf-c25e-4220-82d9-523f1e388e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:44.012825 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d633f2bf-c25e-4220-82d9-523f1e388e23 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:44.013110 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d633f2bf-c25e-4220-82d9-523f1e388e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:44.013401 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d633f2bf-c25e-4220-82d9-523f1e388e23 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:44.014140 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d633f2bf-c25e-4220-82d9-523f1e388e23 None admin] RBAC: Adding query filter params (name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:44.014449 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d633f2bf-c25e-4220-82d9-523f1e388e23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:44.015900 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d633f2bf-c25e-4220-82d9-523f1e388e23 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:44.024960 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:44.030386 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 338/670] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Jun 15 14:29:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:44.030386 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d633f2bf-c25e-4220-82d9-523f1e388e23 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:44.097280 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed6a84-72b0-4981-85aa-7cdf03d59868 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:44.133749 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab0d23-6c6d-46d0-99b3-48265cad1c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:44.134333 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab0d23-6c6d-46d0-99b3-48265cad1c41 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:44.134619 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab0d23-6c6d-46d0-99b3-48265cad1c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:44.134905 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab0d23-6c6d-46d0-99b3-48265cad1c41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:44.137393 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d633f2bf-c25e-4220-82d9-523f1e388e23 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:44.150468 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:44.176146 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27e2be-afd1-467e-94b4-4c50f53fcfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:44.178492 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27e2be-afd1-467e-94b4-4c50f53fcfac None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:44.178492 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27e2be-afd1-467e-94b4-4c50f53fcfac None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:44.178492 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27e2be-afd1-467e-94b4-4c50f53fcfac None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:44.178492 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef27e2be-afd1-467e-94b4-4c50f53fcfac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:44.178492 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef27e2be-afd1-467e-94b4-4c50f53fcfac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:44.179893 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef27e2be-afd1-467e-94b4-4c50f53fcfac None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:44.193300 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 339/671] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:29:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:44.193904 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27e2be-afd1-467e-94b4-4c50f53fcfac None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:44.215572 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab0d23-6c6d-46d0-99b3-48265cad1c41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:44.217167 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab0d23-6c6d-46d0-99b3-48265cad1c41 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:44.253693 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:44.294107 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27e2be-afd1-467e-94b4-4c50f53fcfac None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:44.330624 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2743fd-d0e7-40b7-ae2f-57913d543923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:44.331274 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 333/672] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:44.331386 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2743fd-d0e7-40b7-ae2f-57913d543923 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:44.331675 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2743fd-d0e7-40b7-ae2f-57913d543923 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:44.331954 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2743fd-d0e7-40b7-ae2f-57913d543923 None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:44.332807 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2743fd-d0e7-40b7-ae2f-57913d543923 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:44.333125 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2743fd-d0e7-40b7-ae2f-57913d543923 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:44.334562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2743fd-d0e7-40b7-ae2f-57913d543923 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:44.336441 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab0d23-6c6d-46d0-99b3-48265cad1c41 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:44.347223 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 340/673] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:29:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:44.352152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2743fd-d0e7-40b7-ae2f-57913d543923 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:44.385842 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:44.458880 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab0d23-6c6d-46d0-99b3-48265cad1c41 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:44.467835 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2743fd-d0e7-40b7-ae2f-57913d543923 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:44.495166 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a216ed-0b28-465a-b4ba-127840fc06c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:44.496867 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a216ed-0b28-465a-b4ba-127840fc06c2 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:44.497296 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a216ed-0b28-465a-b4ba-127840fc06c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:44.497710 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a216ed-0b28-465a-b4ba-127840fc06c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:44.500876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:44.504107 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1edff6-a52b-407e-ba31-79e5b6895030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:44.504766 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1edff6-a52b-407e-ba31-79e5b6895030 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:44.505049 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1edff6-a52b-407e-ba31-79e5b6895030 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:44.505322 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1edff6-a52b-407e-ba31-79e5b6895030 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:44.506142 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1edff6-a52b-407e-ba31-79e5b6895030 None admin] RBAC: Adding query filter params (role.id=13dcf68cca9c44888c0fd27a78db3dbe, scope.project.id=d488c79cb1704e9496fc196d8aa6c104, user.id=975ae2790ac849ff88e991231671f510) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:44.506501 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1edff6-a52b-407e-ba31-79e5b6895030 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:44.507983 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1edff6-a52b-407e-ba31-79e5b6895030 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:44.510950 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a216ed-0b28-465a-b4ba-127840fc06c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:44.512435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a216ed-0b28-465a-b4ba-127840fc06c2 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:44.514422 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 341/674] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:29:44 2022] GET /identity/v3/role_assignments?user.id=975ae2790ac849ff88e991231671f510&scope.project.id=d488c79cb1704e9496fc196d8aa6c104&role.id=13dcf68cca9c44888c0fd27a78db3dbe => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:44.515133 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1edff6-a52b-407e-ba31-79e5b6895030 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:44.546920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-89a216ed-0b28-465a-b4ba-127840fc06c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:44.546920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:44.546920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:44.546920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:44.546920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:44.546920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:44.546920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:44.546920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:44.546920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:44.546920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:44.546920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:44.546920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:44.546920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:44.546920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:44.546920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:44.547948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:44.547948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:44.547948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:44.547948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:44.547948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:44.547948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:44.547948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:44.547948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:44.547948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:44.547948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:44.547948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:44.547948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:44.547948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:44.547948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:44.547948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:44.547948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:44.547948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:44.548857 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:44.548857 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:44.548857 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:44.548857 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:44.551744 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 334/675] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:44.552967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a216ed-0b28-465a-b4ba-127840fc06c2 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:44.633683 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:44.642906 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1edff6-a52b-407e-ba31-79e5b6895030 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:44.679688 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a216ed-0b28-465a-b4ba-127840fc06c2 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:44.686770 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c458a-5f03-4a82-ba48-ad1e6ed32079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:44.687776 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c458a-5f03-4a82-ba48-ad1e6ed32079 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:44.688540 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c458a-5f03-4a82-ba48-ad1e6ed32079 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:44.688540 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c458a-5f03-4a82-ba48-ad1e6ed32079 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:44.700590 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c458a-5f03-4a82-ba48-ad1e6ed32079 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:44.702397 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c458a-5f03-4a82-ba48-ad1e6ed32079 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:44.720102 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6250f92e-bfc0-471a-9db4-11f3136b158b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:44.720765 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250f92e-bfc0-471a-9db4-11f3136b158b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:44.721049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250f92e-bfc0-471a-9db4-11f3136b158b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:44.721326 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250f92e-bfc0-471a-9db4-11f3136b158b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:44.722193 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6250f92e-bfc0-471a-9db4-11f3136b158b None admin] RBAC: Adding query filter params (role.id=13dcf68cca9c44888c0fd27a78db3dbe, scope.project.id=d488c79cb1704e9496fc196d8aa6c104, user.id=e4f10214b2bb4eafb3dff7adf71b9bf9) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:44.722508 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6250f92e-bfc0-471a-9db4-11f3136b158b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:44.725633 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6250f92e-bfc0-471a-9db4-11f3136b158b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:44.725751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-8f7c458a-5f03-4a82-ba48-ad1e6ed32079 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:44.725751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:44.725751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:44.725751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:44.725751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:44.725751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:44.725751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:44.725751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:44.725751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:44.725751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:44.725751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:44.725751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:44.725751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:44.725751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:44.725751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:44.726539 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:44.726539 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:44.726539 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:44.726539 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:44.726539 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:44.726539 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:44.726539 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:44.726539 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:44.726539 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:44.726539 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:44.726539 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:44.726539 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:44.726539 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:44.726539 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:44.726539 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:44.726539 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:44.726539 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:44.727448 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:44.727448 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:44.727448 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:44.727448 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:44.727448 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 342/676] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:44.727448 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c458a-5f03-4a82-ba48-ad1e6ed32079 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:44.734791 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 335/677] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:29:44 2022] GET /identity/v3/role_assignments?user.id=e4f10214b2bb4eafb3dff7adf71b9bf9&scope.project.id=d488c79cb1704e9496fc196d8aa6c104&role.id=13dcf68cca9c44888c0fd27a78db3dbe => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:44.736269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51e447-89c2-4285-825d-4b70b7186e68 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:44.736578 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51e447-89c2-4285-825d-4b70b7186e68 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:44.736846 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51e447-89c2-4285-825d-4b70b7186e68 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:44.738231 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 336/678] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:44.738798 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc51e447-89c2-4285-825d-4b70b7186e68 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:44.808726 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:44.862539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc51e447-89c2-4285-825d-4b70b7186e68 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:44.862734 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c458a-5f03-4a82-ba48-ad1e6ed32079 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:44.901430 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a12d23-7284-4cae-96e8-caa5a4a6662a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:44.902578 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a12d23-7284-4cae-96e8-caa5a4a6662a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:44.902864 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a12d23-7284-4cae-96e8-caa5a4a6662a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:44.903055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b346f1-884d-4c33-be16-3c7e579406d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:44.903189 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a12d23-7284-4cae-96e8-caa5a4a6662a None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:44.903686 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b346f1-884d-4c33-be16-3c7e579406d2 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:44.903893 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a12d23-7284-4cae-96e8-caa5a4a6662a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:44.904000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b346f1-884d-4c33-be16-3c7e579406d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:44.904204 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a12d23-7284-4cae-96e8-caa5a4a6662a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:44.904319 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b346f1-884d-4c33-be16-3c7e579406d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:44.905001 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b346f1-884d-4c33-be16-3c7e579406d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:44.905354 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b346f1-884d-4c33-be16-3c7e579406d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:44.905591 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a12d23-7284-4cae-96e8-caa5a4a6662a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:44.906723 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b346f1-884d-4c33-be16-3c7e579406d2 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:44.919384 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 337/679] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:44.920699 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea91f4-d264-4ce3-82d9-dba22d89cc9d None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:44.920990 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea91f4-d264-4ce3-82d9-dba22d89cc9d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:44.921228 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 343/680] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:44.921331 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea91f4-d264-4ce3-82d9-dba22d89cc9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:44.926242 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b346f1-884d-4c33-be16-3c7e579406d2 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:44.937399 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:44.940404 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-2fea91f4-d264-4ce3-82d9-dba22d89cc9d None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:44.941794 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-2fea91f4-d264-4ce3-82d9-dba22d89cc9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:45.029376 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 338/681] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:45.030565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea91f4-d264-4ce3-82d9-dba22d89cc9d None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:45.052659 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b346f1-884d-4c33-be16-3c7e579406d2 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:45.093308 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:45.096937 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e4382-f322-4654-804c-32ecc1c0d783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:45.098142 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e4382-f322-4654-804c-32ecc1c0d783 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:45.098694 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e4382-f322-4654-804c-32ecc1c0d783 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:45.098979 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e4382-f322-4654-804c-32ecc1c0d783 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:45.108827 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8e4382-f322-4654-804c-32ecc1c0d783 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:45.110971 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8e4382-f322-4654-804c-32ecc1c0d783 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:45.129103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-ef8e4382-f322-4654-804c-32ecc1c0d783 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:45.129103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:45.129103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:45.129103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:45.129103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:45.129103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:45.129103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:45.129103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:45.129103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:45.129103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:45.129103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:45.129103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:45.129103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:29:45.129103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:29:45.129103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:29:45.130055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:29:45.130055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:45.130055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:45.130055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:45.130055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:45.130055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:45.130055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:45.130055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:45.130055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:45.130055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:45.130055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:45.130055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:45.130055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:45.130055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:45.130055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:45.130055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:45.130055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:45.131366 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:29:45.131366 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:29:45.131366 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:29:45.131366 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:29:45.131366 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:29:45.131366 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:29:45.131366 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:45.131366 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:45.131366 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 344/682] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:45.135602 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e4382-f322-4654-804c-32ecc1c0d783 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:45.164268 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea91f4-d264-4ce3-82d9-dba22d89cc9d None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:45.209409 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:45.211869 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e8f4f6-2402-41a5-b60d-45bd6841375c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:45.213179 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8f4f6-2402-41a5-b60d-45bd6841375c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:45.213489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8f4f6-2402-41a5-b60d-45bd6841375c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:45.213802 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8f4f6-2402-41a5-b60d-45bd6841375c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:45.223852 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e8f4f6-2402-41a5-b60d-45bd6841375c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:45.226169 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e8f4f6-2402-41a5-b60d-45bd6841375c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:45.249329 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-f6e8f4f6-2402-41a5-b60d-45bd6841375c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:29:45.249329 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:45.249329 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:45.249329 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:45.249329 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:45.249329 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:45.249329 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:45.249329 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:45.249329 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:45.249329 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:45.249329 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:45.249329 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:45.249329 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:29:45.249329 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:29:45.249329 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:29:45.249329 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:29:45.251349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:45.251349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:45.251349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:29:45.251349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:45.251349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:29:45.251349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:45.251349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:45.251349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:45.251349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:45.251349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:45.251349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:45.251349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:45.251349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:45.251349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:45.251349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:45.251349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:45.251349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:45.251349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:45.253512 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:45.253512 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:45.253512 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:29:45.253512 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:29:45.253512 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:29:45.253512 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:29:45.253512 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:29:45.253512 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:29:45.253512 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:29:45.253512 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:45.263190 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e4382-f322-4654-804c-32ecc1c0d783 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:45.263306 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 339/683] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:29:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:45.263306 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e8f4f6-2402-41a5-b60d-45bd6841375c None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:45.291584 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6a8f2-e087-4b14-9558-32a4980dce11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:45.293059 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6a8f2-e087-4b14-9558-32a4980dce11 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:45.293444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6a8f2-e087-4b14-9558-32a4980dce11 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:45.293784 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6a8f2-e087-4b14-9558-32a4980dce11 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:45.305063 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f6a8f2-e087-4b14-9558-32a4980dce11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:45.313479 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-b5f6a8f2-e087-4b14-9558-32a4980dce11 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:29:45.313479 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:45.313479 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:45.313479 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:45.313479 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:45.313479 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:45.313479 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:45.313479 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:45.313479 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:45.313479 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:45.313479 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:45.313479 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:45.313479 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:45.313479 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:45.313479 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:45.313479 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:45.315379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:45.315379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:45.315379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:45.315379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:45.315379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:45.315379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:45.315379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:45.315379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:45.315379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:45.315379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:45.315379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:45.315379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:45.315379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:45.315379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:45.315379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:45.315379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:45.317669 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:45.317669 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:45.317669 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:45.317669 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:45.317669 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:45.317669 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:45.317669 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:45.317669 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:45.317669 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:45.317669 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:45.317669 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:29:45.317669 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:45.317669 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 345/684] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:29:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:45.317669 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6a8f2-e087-4b14-9558-32a4980dce11 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:45.386349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e8f4f6-2402-41a5-b60d-45bd6841375c None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:45.426118 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6a8f2-e087-4b14-9558-32a4980dce11 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:45.430510 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bca066-3712-4aef-ae47-6152da4da36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:45.431228 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bca066-3712-4aef-ae47-6152da4da36d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:45.431514 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bca066-3712-4aef-ae47-6152da4da36d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:45.431797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bca066-3712-4aef-ae47-6152da4da36d None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:45.432589 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bca066-3712-4aef-ae47-6152da4da36d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:45.432903 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bca066-3712-4aef-ae47-6152da4da36d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:45.434980 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bca066-3712-4aef-ae47-6152da4da36d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:45.452421 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 340/685] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:29:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:45.453296 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bca066-3712-4aef-ae47-6152da4da36d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:45.465634 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:45.477457 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67e84d-820f-4391-85f2-ad2ea31b7628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:45.478210 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67e84d-820f-4391-85f2-ad2ea31b7628 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:45.478496 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67e84d-820f-4391-85f2-ad2ea31b7628 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:45.478813 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67e84d-820f-4391-85f2-ad2ea31b7628 None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:45.479688 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67e84d-820f-4391-85f2-ad2ea31b7628 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:45.480551 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67e84d-820f-4391-85f2-ad2ea31b7628 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:45.482060 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67e84d-820f-4391-85f2-ad2ea31b7628 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:45.499397 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:45.505380 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 346/686] 10.222.0.45 () {56 vars in 1135 bytes} [Wed Jun 15 14:29:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:45.506083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67e84d-820f-4391-85f2-ad2ea31b7628 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:45.585788 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bca066-3712-4aef-ae47-6152da4da36d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:45.615426 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67e84d-820f-4391-85f2-ad2ea31b7628 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:45.627091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d90253-df1e-466a-aa40-883e499dbbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:45.627944 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d90253-df1e-466a-aa40-883e499dbbe2 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:45.628288 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d90253-df1e-466a-aa40-883e499dbbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:45.628663 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d90253-df1e-466a-aa40-883e499dbbe2 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:45.629509 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d90253-df1e-466a-aa40-883e499dbbe2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:45.630032 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d90253-df1e-466a-aa40-883e499dbbe2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:45.631425 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d90253-df1e-466a-aa40-883e499dbbe2 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:45.644725 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 341/687] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:45.644725 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d90253-df1e-466a-aa40-883e499dbbe2 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:45.665448 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9359e71b-c7ea-4650-9e3e-7d8360bcf0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:45.666630 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359e71b-c7ea-4650-9e3e-7d8360bcf0ee None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:45.667106 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359e71b-c7ea-4650-9e3e-7d8360bcf0ee None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:45.667727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359e71b-c7ea-4650-9e3e-7d8360bcf0ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:45.669068 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9359e71b-c7ea-4650-9e3e-7d8360bcf0ee None admin] RBAC: Adding query filter params (role.id=13dcf68cca9c44888c0fd27a78db3dbe, scope.project.id=d488c79cb1704e9496fc196d8aa6c104, user.id=b47cec76cb1c4afb8e011ff04a885f96) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:45.669804 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9359e71b-c7ea-4650-9e3e-7d8360bcf0ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:45.671949 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9359e71b-c7ea-4650-9e3e-7d8360bcf0ee None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:45.679592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:45.684939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 347/688] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:29:45 2022] GET /identity/v3/role_assignments?user.id=b47cec76cb1c4afb8e011ff04a885f96&scope.project.id=d488c79cb1704e9496fc196d8aa6c104&role.id=13dcf68cca9c44888c0fd27a78db3dbe => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:45.684939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9359e71b-c7ea-4650-9e3e-7d8360bcf0ee None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:45.788502 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:45.818819 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d90253-df1e-466a-aa40-883e499dbbe2 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:45.884729 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9359e71b-c7ea-4650-9e3e-7d8360bcf0ee None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:45.888447 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09178a69-49ef-49b0-8649-ffd9b9d64e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:45.889117 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09178a69-49ef-49b0-8649-ffd9b9d64e34 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:45.889399 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09178a69-49ef-49b0-8649-ffd9b9d64e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:45.889674 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09178a69-49ef-49b0-8649-ffd9b9d64e34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:45.939836 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c39b0d-8c6b-458b-8428-a05f3ec26122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:45.939836 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c39b0d-8c6b-458b-8428-a05f3ec26122 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:45.940117 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c39b0d-8c6b-458b-8428-a05f3ec26122 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:45.940347 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c39b0d-8c6b-458b-8428-a05f3ec26122 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:45.954140 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c39b0d-8c6b-458b-8428-a05f3ec26122 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:45.955670 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c39b0d-8c6b-458b-8428-a05f3ec26122 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:45.976773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-62c39b0d-8c6b-458b-8428-a05f3ec26122 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:45.976773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:45.976773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:45.976773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:45.976773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:45.976773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:45.976773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:45.976773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:45.976773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:45.976773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:45.976773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:45.976773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:45.976773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:45.976773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:45.976773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:45.977707 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:45.977707 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:45.977707 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:45.977707 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:45.977707 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:45.977707 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:45.977707 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:45.977707 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:45.977707 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:45.977707 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:45.977707 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:45.977707 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:45.977707 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:45.977707 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:45.977707 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:45.977707 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:45.977707 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:45.978592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:45.978592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:45.978592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:45.978592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:45.978592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 348/689] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:45.980785 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09178a69-49ef-49b0-8649-ffd9b9d64e34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:45.982479 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09178a69-49ef-49b0-8649-ffd9b9d64e34 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:45.983851 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c39b0d-8c6b-458b-8428-a05f3ec26122 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:46.023398 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:46.095319 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:46.098216 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c39b0d-8c6b-458b-8428-a05f3ec26122 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:46.098928 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 342/690] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:46.107105 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09178a69-49ef-49b0-8649-ffd9b9d64e34 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:46.142326 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad75bd0f-58a6-4fca-a197-5dc4331b7f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:46.143059 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75bd0f-58a6-4fca-a197-5dc4331b7f49 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:46.143306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75bd0f-58a6-4fca-a197-5dc4331b7f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:46.143583 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75bd0f-58a6-4fca-a197-5dc4331b7f49 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:46.145189 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad75bd0f-58a6-4fca-a197-5dc4331b7f49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:46.145189 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad75bd0f-58a6-4fca-a197-5dc4331b7f49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:46.151017 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad75bd0f-58a6-4fca-a197-5dc4331b7f49 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:46.158341 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 349/691] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:46.163741 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad75bd0f-58a6-4fca-a197-5dc4331b7f49 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:46.276123 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09178a69-49ef-49b0-8649-ffd9b9d64e34 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:46.307019 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:46.324357 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4942c4-c2e3-4991-a58e-2723b5eca537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:46.325482 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4942c4-c2e3-4991-a58e-2723b5eca537 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:46.325967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4942c4-c2e3-4991-a58e-2723b5eca537 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:46.326473 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4942c4-c2e3-4991-a58e-2723b5eca537 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:46.335087 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad75bd0f-58a6-4fca-a197-5dc4331b7f49 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:46.359893 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4942c4-c2e3-4991-a58e-2723b5eca537 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:46.361368 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4942c4-c2e3-4991-a58e-2723b5eca537 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:46.371350 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46687140-4a38-4859-a000-39e08c4602f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:46.372142 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46687140-4a38-4859-a000-39e08c4602f0 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:46.372428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46687140-4a38-4859-a000-39e08c4602f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:46.372700 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46687140-4a38-4859-a000-39e08c4602f0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:46.381208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46687140-4a38-4859-a000-39e08c4602f0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:46.382744 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46687140-4a38-4859-a000-39e08c4602f0 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:46.400962 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-46687140-4a38-4859-a000-39e08c4602f0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:29:46.400962 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:46.400962 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:46.400962 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:46.400962 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:46.400962 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:46.400962 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:46.400962 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:46.400962 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:46.400962 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:46.400962 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:46.400962 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:46.400962 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:29:46.400962 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:29:46.400962 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:29:46.400962 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:29:46.401948 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:46.401948 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:46.401948 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:29:46.401948 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:46.401948 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:29:46.401948 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:46.401948 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:46.401948 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:46.401948 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:46.401948 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:46.401948 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:46.401948 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:46.401948 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:46.401948 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:46.401948 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:46.401948 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:46.401948 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:46.401948 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:46.403058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:46.403058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:46.403058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:29:46.403058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:29:46.403058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:29:46.403058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:29:46.403058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:29:46.403058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:29:46.403058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:29:46.403058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:46.403058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 350/692] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Jun 15 14:29:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:46.406043 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-dc4942c4-c2e3-4991-a58e-2723b5eca537 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:46.406043 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:46.406043 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:46.406043 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:46.406043 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:46.406043 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:46.406043 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:46.406043 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:46.406043 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:46.406043 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:46.406043 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:46.406043 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:46.406043 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:46.406043 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:46.406043 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:46.406911 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:46.406911 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:46.406911 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:46.406911 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:46.406911 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:46.406911 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:46.406911 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:46.406911 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:46.406911 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:46.406911 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:46.406911 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:46.406911 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:46.406911 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:46.406911 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:46.406911 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:46.406911 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:46.406911 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:46.407805 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:46.407805 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:46.407805 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:46.407805 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:46.411063 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 343/693] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:29:46.415076 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46687140-4a38-4859-a000-39e08c4602f0 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:46.416150 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4942c4-c2e3-4991-a58e-2723b5eca537 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:46.462194 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:46.603223 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4942c4-c2e3-4991-a58e-2723b5eca537 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:46.626802 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46687140-4a38-4859-a000-39e08c4602f0 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:46.643964 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67760cc9-b847-432d-93ee-e4950d504a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:46.644907 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67760cc9-b847-432d-93ee-e4950d504a2c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:46.645193 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67760cc9-b847-432d-93ee-e4950d504a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:46.645469 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67760cc9-b847-432d-93ee-e4950d504a2c None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:46.646304 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67760cc9-b847-432d-93ee-e4950d504a2c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:46.646616 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67760cc9-b847-432d-93ee-e4950d504a2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:46.648067 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67760cc9-b847-432d-93ee-e4950d504a2c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:46.666373 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 344/694] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Jun 15 14:29:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:46.673190 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67760cc9-b847-432d-93ee-e4950d504a2c None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:46.714967 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5b161-f238-432f-9e7b-eaa9945a5556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:46.716603 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5b161-f238-432f-9e7b-eaa9945a5556 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:46.717265 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5b161-f238-432f-9e7b-eaa9945a5556 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:46.719540 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5b161-f238-432f-9e7b-eaa9945a5556 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:46.720485 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d5b161-f238-432f-9e7b-eaa9945a5556 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:46.721016 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d5b161-f238-432f-9e7b-eaa9945a5556 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:46.721839 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:46.724240 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d5b161-f238-432f-9e7b-eaa9945a5556 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:46.750177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 351/695] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:46.757487 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5b161-f238-432f-9e7b-eaa9945a5556 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:46.809463 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67760cc9-b847-432d-93ee-e4950d504a2c None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:46.865254 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8663aac6-c4e6-458e-8d96-f0806d64539b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:46.865888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663aac6-c4e6-458e-8d96-f0806d64539b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:46.866180 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663aac6-c4e6-458e-8d96-f0806d64539b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:46.866461 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663aac6-c4e6-458e-8d96-f0806d64539b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:46.930479 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:46.957772 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5b161-f238-432f-9e7b-eaa9945a5556 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:46.958259 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8663aac6-c4e6-458e-8d96-f0806d64539b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:46.960037 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8663aac6-c4e6-458e-8d96-f0806d64539b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:47.002703 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c53f49-df80-42d2-a5c8-dbb75b559e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:47.003918 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c53f49-df80-42d2-a5c8-dbb75b559e6d None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:47.004625 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:47.005122 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c53f49-df80-42d2-a5c8-dbb75b559e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:47.007142 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c53f49-df80-42d2-a5c8-dbb75b559e6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:47.017272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c53f49-df80-42d2-a5c8-dbb75b559e6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:47.020045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c53f49-df80-42d2-a5c8-dbb75b559e6d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:47.038607 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-b0c53f49-df80-42d2-a5c8-dbb75b559e6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:47.038607 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:47.038607 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:47.038607 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:47.038607 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:47.038607 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:47.038607 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:47.038607 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:47.038607 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:47.038607 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:47.038607 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:47.038607 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:47.038607 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:29:47.038607 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:29:47.038607 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:29:47.039592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:29:47.039592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:47.039592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:47.039592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:47.039592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:47.039592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:47.039592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:47.039592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:47.039592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:47.039592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:47.039592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:47.039592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:47.039592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:47.039592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:47.039592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:47.039592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:47.039592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:47.040464 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:29:47.040464 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:29:47.040464 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:29:47.040464 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:29:47.040464 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:29:47.040464 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:29:47.040464 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:47.040464 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:47.040464 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 352/696] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:29:47.043502 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b0857-e945-41df-b8e3-cd0348f98556 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:47.044470 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b0857-e945-41df-b8e3-cd0348f98556 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:47.045432 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b0857-e945-41df-b8e3-cd0348f98556 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:47.047223 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 353/697] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:47.048497 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b0857-e945-41df-b8e3-cd0348f98556 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:47.088032 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 345/698] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:47.090675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2e31b-ffdb-4c1a-aa8b-1346ec29db09 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:47.090961 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2e31b-ffdb-4c1a-aa8b-1346ec29db09 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:47.091262 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2e31b-ffdb-4c1a-aa8b-1346ec29db09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:47.111854 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-72e2e31b-ffdb-4c1a-aa8b-1346ec29db09 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:47.113103 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-72e2e31b-ffdb-4c1a-aa8b-1346ec29db09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:47.177145 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b0857-e945-41df-b8e3-cd0348f98556 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:47.188130 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 346/699] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:47.189047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2e31b-ffdb-4c1a-aa8b-1346ec29db09 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:47.220155 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab21874-c1ad-4422-a5fb-8c122eee5734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:47.220821 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab21874-c1ad-4422-a5fb-8c122eee5734 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:47.221109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab21874-c1ad-4422-a5fb-8c122eee5734 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:47.221388 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab21874-c1ad-4422-a5fb-8c122eee5734 None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:47.222179 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab21874-c1ad-4422-a5fb-8c122eee5734 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:47.222499 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab21874-c1ad-4422-a5fb-8c122eee5734 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:47.223967 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab21874-c1ad-4422-a5fb-8c122eee5734 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:47.234050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:47.240597 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 354/700] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:29:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:47.242171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c46ee4-db00-4ec1-b166-71c711dfef5e None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:47.242672 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c46ee4-db00-4ec1-b166-71c711dfef5e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:47.242811 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c46ee4-db00-4ec1-b166-71c711dfef5e None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:47.248108 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 355/701] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:47.248108 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c46ee4-db00-4ec1-b166-71c711dfef5e None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:47.266839 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:47.312538 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2e31b-ffdb-4c1a-aa8b-1346ec29db09 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:47.346491 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659758ba-b617-4f37-9189-70810c1d5b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:47.347316 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659758ba-b617-4f37-9189-70810c1d5b5c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:47.347597 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659758ba-b617-4f37-9189-70810c1d5b5c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:47.347870 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659758ba-b617-4f37-9189-70810c1d5b5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:47.358689 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c46ee4-db00-4ec1-b166-71c711dfef5e None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:47.358859 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659758ba-b617-4f37-9189-70810c1d5b5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:47.360248 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659758ba-b617-4f37-9189-70810c1d5b5c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:47.378491 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-659758ba-b617-4f37-9189-70810c1d5b5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:47.378491 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:47.378491 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:47.378491 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:47.378491 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:47.378491 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:47.378491 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:47.378491 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:47.378491 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:47.378491 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:47.378491 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:47.378491 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:47.378491 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:47.378491 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:47.378491 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:47.379455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:47.379455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:47.379455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:47.379455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:47.379455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:47.379455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:47.379455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:47.379455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:47.379455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:47.379455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:47.379455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:47.379455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:47.379455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:47.379455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:47.379455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:47.379455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:47.379455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:47.385335 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:47.385335 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:47.385335 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:47.385335 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:47.385335 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 347/702] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:47.385335 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659758ba-b617-4f37-9189-70810c1d5b5c None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:47.404606 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ee1e69-391c-471e-92bc-d177197dc353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:47.405532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee1e69-391c-471e-92bc-d177197dc353 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:47.405933 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee1e69-391c-471e-92bc-d177197dc353 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:47.406280 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee1e69-391c-471e-92bc-d177197dc353 None admin] PATH_INFO: `/v3/services/network` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:47.407578 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ee1e69-391c-471e-92bc-d177197dc353 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:47.409413 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ee1e69-391c-471e-92bc-d177197dc353 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:47.423885 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-b8ee1e69-391c-471e-92bc-d177197dc353 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:29:47.423885 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:47.423885 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:29:47.423885 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:29:47.423885 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:29:47.423885 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:29:47.423885 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:29:47.423885 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:29:47.423885 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:29:47.423885 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:47.423885 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:29:47.423885 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:47.423885 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:47.423885 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:47.423885 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:47.423885 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:47.423885 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:47.425105 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:47.425105 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:47.425105 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:47.425105 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:47.425105 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:47.425105 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:47.425105 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:29:47.425105 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:29:47.425105 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:29:47.425105 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:29:47.425105 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:47.425105 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:47.425105 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:47.425105 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:47.425105 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:47.425105 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:47.426151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:47.426151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:47.426151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:47.426151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:47.426151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:47.426151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:47.426151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:47.426151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:47.426151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:47.426151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:47.426151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:29:47.426151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:29:47.426151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:29:47.426151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:47.427637 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 356/703] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:47 2022] GET /identity/v3/services/network => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:47.428489 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c90ff2-6696-4c92-9082-546a4df8d67d None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:47.428819 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c90ff2-6696-4c92-9082-546a4df8d67d None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:47.429164 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c90ff2-6696-4c92-9082-546a4df8d67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:47.447113 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-75c90ff2-6696-4c92-9082-546a4df8d67d None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:47.448289 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-75c90ff2-6696-4c92-9082-546a4df8d67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:47.498167 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659758ba-b617-4f37-9189-70810c1d5b5c None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:47.510551 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:47.518081 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:47.520104 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 357/704] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:47.520947 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c90ff2-6696-4c92-9082-546a4df8d67d None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:47.546854 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a191e9c-f0d5-4100-a92c-111a30d41184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:47.547675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a191e9c-f0d5-4100-a92c-111a30d41184 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:47.548066 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a191e9c-f0d5-4100-a92c-111a30d41184 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:47.548436 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a191e9c-f0d5-4100-a92c-111a30d41184 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:47.549368 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a191e9c-f0d5-4100-a92c-111a30d41184 None admin] RBAC: Adding query filter params (role.id=13dcf68cca9c44888c0fd27a78db3dbe, scope.project.id=d488c79cb1704e9496fc196d8aa6c104, user.id=641a81b3140f47119b5c2323a97a6681) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:47.549787 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a191e9c-f0d5-4100-a92c-111a30d41184 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:47.551468 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a191e9c-f0d5-4100-a92c-111a30d41184 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:47.557481 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 348/705] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:29:47 2022] GET /identity/v3/role_assignments?user.id=641a81b3140f47119b5c2323a97a6681&scope.project.id=d488c79cb1704e9496fc196d8aa6c104&role.id=13dcf68cca9c44888c0fd27a78db3dbe => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:47.558351 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a191e9c-f0d5-4100-a92c-111a30d41184 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:47.636936 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c90ff2-6696-4c92-9082-546a4df8d67d None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:47.677504 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa37e0a-eb21-43c4-9fa1-61dbf01788fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:47.678229 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa37e0a-eb21-43c4-9fa1-61dbf01788fc None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:47.678456 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa37e0a-eb21-43c4-9fa1-61dbf01788fc None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:47.678791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa37e0a-eb21-43c4-9fa1-61dbf01788fc None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:47.679635 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa37e0a-eb21-43c4-9fa1-61dbf01788fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:47.679996 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa37e0a-eb21-43c4-9fa1-61dbf01788fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:47.681426 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa37e0a-eb21-43c4-9fa1-61dbf01788fc None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:47.700343 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 358/706] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:47.700343 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa37e0a-eb21-43c4-9fa1-61dbf01788fc None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:47.721247 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a191e9c-f0d5-4100-a92c-111a30d41184 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:47.768034 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:47.771088 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4594148-b2b3-4d5c-baa7-c1679c034907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:47.771970 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4594148-b2b3-4d5c-baa7-c1679c034907 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:47.772419 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4594148-b2b3-4d5c-baa7-c1679c034907 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:47.772864 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4594148-b2b3-4d5c-baa7-c1679c034907 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:47.773734 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4594148-b2b3-4d5c-baa7-c1679c034907 None admin] RBAC: Adding query filter params (name=network) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:47.774471 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4594148-b2b3-4d5c-baa7-c1679c034907 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:47.777460 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4594148-b2b3-4d5c-baa7-c1679c034907 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:47.785777 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 349/707] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Jun 15 14:29:47 2022] GET /identity/v3/services?name=network => generated 124 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:47.786478 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4594148-b2b3-4d5c-baa7-c1679c034907 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:47.831665 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa37e0a-eb21-43c4-9fa1-61dbf01788fc None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:47.847304 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:47.876657 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3893a71e-0529-4c58-9fa4-23e40d5335f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:47.876657 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893a71e-0529-4c58-9fa4-23e40d5335f2 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:47.876657 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893a71e-0529-4c58-9fa4-23e40d5335f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:47.876657 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893a71e-0529-4c58-9fa4-23e40d5335f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:47.887751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3893a71e-0529-4c58-9fa4-23e40d5335f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:47.889230 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3893a71e-0529-4c58-9fa4-23e40d5335f2 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:47.909487 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-3893a71e-0529-4c58-9fa4-23e40d5335f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:47.909487 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:47.909487 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:47.909487 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:47.909487 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:47.909487 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:47.909487 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:47.909487 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:47.909487 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:47.909487 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:47.909487 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:47.909487 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:47.909487 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:47.909487 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:47.909487 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:47.910444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:47.910444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:47.910444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:47.910444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:47.910444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:47.910444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:47.910444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:47.910444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:47.910444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:47.910444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:47.910444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:47.910444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:47.910444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:47.910444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:47.910444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:47.910444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:47.910444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:47.911386 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:47.911386 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:47.911386 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:47.911386 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:47.911611 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 359/708] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:47.916557 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3893a71e-0529-4c58-9fa4-23e40d5335f2 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:47.916697 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4594148-b2b3-4d5c-baa7-c1679c034907 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:47.956214 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b150a6-3b79-4a59-8d51-b9ced54a13b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:47.956991 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b150a6-3b79-4a59-8d51-b9ced54a13b6 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:47.957274 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b150a6-3b79-4a59-8d51-b9ced54a13b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:47.957655 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b150a6-3b79-4a59-8d51-b9ced54a13b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:47.967013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b150a6-3b79-4a59-8d51-b9ced54a13b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:47.969095 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b150a6-3b79-4a59-8d51-b9ced54a13b6 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:47.986436 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-e1b150a6-3b79-4a59-8d51-b9ced54a13b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:47.986436 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:47.986436 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:47.986436 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:47.986436 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:47.986436 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:47.986436 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:47.986436 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:47.986436 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:47.986436 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:47.986436 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:47.986436 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:47.986436 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:29:47.986436 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:29:47.986436 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:29:47.987403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:29:47.987403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:47.987403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:47.987403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:47.987403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:47.987403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:47.987403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:47.987403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:47.987403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:47.987403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:47.987403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:47.987403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:47.987403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:47.987403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:47.987403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:47.987403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:47.987403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:47.988293 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:29:47.988293 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:29:47.988293 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:29:47.988293 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:29:47.988293 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:29:47.988293 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:29:47.988293 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:47.988293 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:47.988293 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 350/709] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:47.989224 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b150a6-3b79-4a59-8d51-b9ced54a13b6 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:48.028839 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3893a71e-0529-4c58-9fa4-23e40d5335f2 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:48.036284 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:48.069566 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b75a925-9cb3-4ef9-9ae0-af8d4ab47a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:48.070321 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75a925-9cb3-4ef9-9ae0-af8d4ab47a65 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:48.070574 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75a925-9cb3-4ef9-9ae0-af8d4ab47a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:48.070855 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75a925-9cb3-4ef9-9ae0-af8d4ab47a65 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:48.071656 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b75a925-9cb3-4ef9-9ae0-af8d4ab47a65 None admin] RBAC: Adding query filter params (type=network) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:48.071969 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b75a925-9cb3-4ef9-9ae0-af8d4ab47a65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:48.074096 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b75a925-9cb3-4ef9-9ae0-af8d4ab47a65 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:48.087024 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 360/710] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Jun 15 14:29:47 2022] GET /identity/v3/services?type=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:48.087024 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b75a925-9cb3-4ef9-9ae0-af8d4ab47a65 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:48.136589 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b150a6-3b79-4a59-8d51-b9ced54a13b6 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:48.139838 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:48.197251 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b75a925-9cb3-4ef9-9ae0-af8d4ab47a65 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:48.206785 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e601afc8-0430-409a-b6dc-2c9b7cee679f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:48.207423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601afc8-0430-409a-b6dc-2c9b7cee679f None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:48.207710 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601afc8-0430-409a-b6dc-2c9b7cee679f None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:48.207986 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601afc8-0430-409a-b6dc-2c9b7cee679f None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:48.208729 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e601afc8-0430-409a-b6dc-2c9b7cee679f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:48.209037 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e601afc8-0430-409a-b6dc-2c9b7cee679f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:48.210537 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e601afc8-0430-409a-b6dc-2c9b7cee679f None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:48.229688 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 351/711] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:48.235665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e601afc8-0430-409a-b6dc-2c9b7cee679f None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:48.238097 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd3467-c959-4af3-96a3-5e754150afcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:48.238981 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd3467-c959-4af3-96a3-5e754150afcb None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:48.239500 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd3467-c959-4af3-96a3-5e754150afcb None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:48.239915 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd3467-c959-4af3-96a3-5e754150afcb None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:48.240915 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd3467-c959-4af3-96a3-5e754150afcb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:48.241621 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd3467-c959-4af3-96a3-5e754150afcb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:48.243393 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd3467-c959-4af3-96a3-5e754150afcb None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:48.259498 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 361/712] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:29:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:48.277937 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd3467-c959-4af3-96a3-5e754150afcb None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:48.326700 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:48.371156 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e601afc8-0430-409a-b6dc-2c9b7cee679f None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:48.411184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477e23cf-8dbf-42c9-97a5-aac888d98553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:48.411566 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd3467-c959-4af3-96a3-5e754150afcb None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:48.412580 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e23cf-8dbf-42c9-97a5-aac888d98553 None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:48.412580 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e23cf-8dbf-42c9-97a5-aac888d98553 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:48.412580 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e23cf-8dbf-42c9-97a5-aac888d98553 None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:48.413284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477e23cf-8dbf-42c9-97a5-aac888d98553 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:48.415822 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477e23cf-8dbf-42c9-97a5-aac888d98553 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:48.447537 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 352/713] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Jun 15 14:29:48 2022] POST /identity/v3/users => generated 254 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:29:48.449422 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe99dbc-8184-4512-a597-7ff8fce3019f None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:48.449765 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe99dbc-8184-4512-a597-7ff8fce3019f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:48.450043 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe99dbc-8184-4512-a597-7ff8fce3019f None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:48.457219 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4723011c-1f9a-4a1f-b195-40cf2afdada3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:48.457828 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723011c-1f9a-4a1f-b195-40cf2afdada3 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:48.458112 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723011c-1f9a-4a1f-b195-40cf2afdada3 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:48.458442 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723011c-1f9a-4a1f-b195-40cf2afdada3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:48.459423 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4723011c-1f9a-4a1f-b195-40cf2afdada3 None admin] RBAC: Adding query filter params (role.id=ed1cfa2fcf724a4d84f5acea100930c7, scope.project.id=d488c79cb1704e9496fc196d8aa6c104, user.id=f3185bd2458c4e22830975700888e9be) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:48.459742 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4723011c-1f9a-4a1f-b195-40cf2afdada3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:48.461342 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4723011c-1f9a-4a1f-b195-40cf2afdada3 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:48.463911 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 353/714] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:48 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:48.470752 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac32e4-adda-43cb-80bb-76121fba5b6a None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:48.471085 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac32e4-adda-43cb-80bb-76121fba5b6a None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:48.471370 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac32e4-adda-43cb-80bb-76121fba5b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:48.475208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 362/715] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:29:48 2022] GET /identity/v3/role_assignments?user.id=f3185bd2458c4e22830975700888e9be&scope.project.id=d488c79cb1704e9496fc196d8aa6c104&role.id=ed1cfa2fcf724a4d84f5acea100930c7 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:48.491319 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-8fac32e4-adda-43cb-80bb-76121fba5b6a None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:48.492567 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-8fac32e4-adda-43cb-80bb-76121fba5b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:48.512163 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:48.583157 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 354/716] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:29:48.599483 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4723011c-1f9a-4a1f-b195-40cf2afdada3 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:48.740635 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:48.747938 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4723011c-1f9a-4a1f-b195-40cf2afdada3 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:48.784589 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220bec6-ae99-4565-a6fa-376945a129ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:48.785463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220bec6-ae99-4565-a6fa-376945a129ec None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:48.785749 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220bec6-ae99-4565-a6fa-376945a129ec None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:48.786070 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220bec6-ae99-4565-a6fa-376945a129ec None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:48.786737 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6220bec6-ae99-4565-a6fa-376945a129ec None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:48.788148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6220bec6-ae99-4565-a6fa-376945a129ec None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:48.801117 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-6220bec6-ae99-4565-a6fa-376945a129ec None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:29:48.801117 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:48.801117 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:29:48.801117 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:29:48.801117 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:29:48.801117 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:29:48.801117 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:29:48.801117 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:29:48.801117 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:29:48.801117 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:48.801117 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:29:48.801117 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:48.801117 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:48.801117 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:48.801117 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:48.801117 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:48.802050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:48.802050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:48.802050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:48.802050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:48.802050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:48.802050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:48.802050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:48.802050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:29:48.802050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:29:48.802050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:29:48.802050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:29:48.802050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:48.802050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:48.802050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:48.802050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:48.802050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:48.802881 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:48.802881 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:48.802881 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:48.802881 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:48.802881 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:48.802881 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:48.802881 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:48.802881 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:48.802881 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:48.802881 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:48.802881 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:48.802881 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:29:48.802881 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:29:48.802881 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:29:48.802881 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:48.802881 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 363/717] 10.222.0.45 () {56 vars in 1129 bytes} [Wed Jun 15 14:29:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:48.808797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac32e4-adda-43cb-80bb-76121fba5b6a None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:48.912353 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac32e4-adda-43cb-80bb-76121fba5b6a None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:48.950568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0345f23-e887-44ae-8223-e24436125d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:48.951336 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0345f23-e887-44ae-8223-e24436125d78 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:48.951622 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0345f23-e887-44ae-8223-e24436125d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:48.951897 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0345f23-e887-44ae-8223-e24436125d78 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:48.952610 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0345f23-e887-44ae-8223-e24436125d78 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:48.952921 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0345f23-e887-44ae-8223-e24436125d78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:48.954292 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0345f23-e887-44ae-8223-e24436125d78 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:48.961492 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 355/718] 10.222.0.45 () {56 vars in 1124 bytes} [Wed Jun 15 14:29:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:48.965095 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220bec6-ae99-4565-a6fa-376945a129ec None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:49.072616 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220bec6-ae99-4565-a6fa-376945a129ec None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:49.109371 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a69ff2-2857-4f51-9abb-7aaad7715a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:49.110047 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a69ff2-2857-4f51-9abb-7aaad7715a29 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:49.110331 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a69ff2-2857-4f51-9abb-7aaad7715a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:49.110608 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a69ff2-2857-4f51-9abb-7aaad7715a29 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:49.111347 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a69ff2-2857-4f51-9abb-7aaad7715a29 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:49.111658 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a69ff2-2857-4f51-9abb-7aaad7715a29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:49.113042 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a69ff2-2857-4f51-9abb-7aaad7715a29 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:49.123148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 364/719] 10.222.0.45 () {56 vars in 1124 bytes} [Wed Jun 15 14:29:48 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:50.176060 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce8ce2-b250-481b-b4f8-bb53650a14c5 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:50.176751 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce8ce2-b250-481b-b4f8-bb53650a14c5 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:50.176751 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce8ce2-b250-481b-b4f8-bb53650a14c5 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:50.178426 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 356/720] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:50.183262 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ba764-205c-484c-ba71-be09068fe04e None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:50.183736 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ba764-205c-484c-ba71-be09068fe04e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:50.183883 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ba764-205c-484c-ba71-be09068fe04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:50.203990 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-6f3ba764-205c-484c-ba71-be09068fe04e None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:50.205200 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6f3ba764-205c-484c-ba71-be09068fe04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:50.227612 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:50.293503 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 365/721] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:50.331838 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce8ce2-b250-481b-b4f8-bb53650a14c5 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:50.348558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:50.457055 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce8ce2-b250-481b-b4f8-bb53650a14c5 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:50.514950 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecde30e-079e-493d-8879-3263733624fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:50.515785 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecde30e-079e-493d-8879-3263733624fc None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:50.516065 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecde30e-079e-493d-8879-3263733624fc None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:50.516338 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecde30e-079e-493d-8879-3263733624fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:50.540691 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecde30e-079e-493d-8879-3263733624fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:50.548557 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-4ecde30e-079e-493d-8879-3263733624fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:29:50.548557 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:50.548557 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:50.548557 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:50.548557 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:50.548557 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:50.548557 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:50.548557 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:50.548557 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:50.548557 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:50.548557 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:50.548557 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:50.548557 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:50.548557 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:50.548557 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:50.548557 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:50.549445 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:50.549445 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:50.549445 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:50.549445 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:50.549445 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:50.549445 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:50.549445 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:50.549445 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:50.549445 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:50.549445 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:50.549445 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:50.549445 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:50.549445 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:50.549445 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:50.549445 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:50.549445 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:50.550260 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:50.550260 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:50.550260 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:50.550260 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:50.550260 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:50.550260 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:50.550260 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:50.550260 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:50.550260 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:50.550260 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:50.550260 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:29:50.550260 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:50.550260 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 357/722] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:29:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:50.560088 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ba764-205c-484c-ba71-be09068fe04e None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:50.695662 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ba764-205c-484c-ba71-be09068fe04e None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:50.760367 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a22e7de-65cf-488e-8620-6a2a71a29356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:50.761055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22e7de-65cf-488e-8620-6a2a71a29356 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:50.761385 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22e7de-65cf-488e-8620-6a2a71a29356 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:50.761667 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22e7de-65cf-488e-8620-6a2a71a29356 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:50.762429 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a22e7de-65cf-488e-8620-6a2a71a29356 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:50.762741 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a22e7de-65cf-488e-8620-6a2a71a29356 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:50.764159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a22e7de-65cf-488e-8620-6a2a71a29356 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:50.775955 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 366/723] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:50.784178 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecde30e-079e-493d-8879-3263733624fc None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:50.860613 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06b27c-1671-4a67-9a41-d29a76eab64d None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:50.860938 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06b27c-1671-4a67-9a41-d29a76eab64d None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:50.861174 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06b27c-1671-4a67-9a41-d29a76eab64d None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:50.862946 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 367/724] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:50.873474 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da40ac-1751-4c6c-bfa9-b33f6d20caa7 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:50.873764 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da40ac-1751-4c6c-bfa9-b33f6d20caa7 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:50.874045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da40ac-1751-4c6c-bfa9-b33f6d20caa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:50.899728 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:50.905691 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-50da40ac-1751-4c6c-bfa9-b33f6d20caa7 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:50.908480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-50da40ac-1751-4c6c-bfa9-b33f6d20caa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:50.927299 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:50.935541 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecde30e-079e-493d-8879-3263733624fc None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:50.993976 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a3d68f-4162-487d-b3ae-bd78ce812c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:50.995673 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3d68f-4162-487d-b3ae-bd78ce812c0c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:50.996260 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3d68f-4162-487d-b3ae-bd78ce812c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:50.998067 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3d68f-4162-487d-b3ae-bd78ce812c0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:51.016605 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a3d68f-4162-487d-b3ae-bd78ce812c0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:51.018915 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a3d68f-4162-487d-b3ae-bd78ce812c0c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:51.035182 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 368/725] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:29:51.046193 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-f6a3d68f-4162-487d-b3ae-bd78ce812c0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:51.046193 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:51.046193 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:51.046193 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:51.046193 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:51.046193 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:51.046193 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:51.046193 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:51.046193 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:51.046193 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:51.046193 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:51.046193 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:51.046193 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:51.046193 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:51.046193 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:51.047380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:51.047380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:51.047380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:51.047380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:51.047380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:51.047380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:51.047380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:51.047380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:51.047380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:51.047380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:51.047380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:51.047380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:51.047380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:51.047380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:51.047380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:51.047380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:51.047380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:51.048464 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:51.048464 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:51.048464 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:51.048464 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:51.050257 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 358/726] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:51.055646 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da40ac-1751-4c6c-bfa9-b33f6d20caa7 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:51.066176 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a3d68f-4162-487d-b3ae-bd78ce812c0c None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:51.202653 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da40ac-1751-4c6c-bfa9-b33f6d20caa7 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:51.245521 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ff8c9-b3cc-40a1-8fd3-2760a0c45887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:51.247244 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ff8c9-b3cc-40a1-8fd3-2760a0c45887 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:51.247733 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ff8c9-b3cc-40a1-8fd3-2760a0c45887 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:51.248212 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ff8c9-b3cc-40a1-8fd3-2760a0c45887 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:51.249189 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ff8c9-b3cc-40a1-8fd3-2760a0c45887 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:51.249747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ff8c9-b3cc-40a1-8fd3-2760a0c45887 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:51.251428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ff8c9-b3cc-40a1-8fd3-2760a0c45887 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:51.257910 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a3d68f-4162-487d-b3ae-bd78ce812c0c None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:51.265272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 369/727] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:51.276089 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ff8c9-b3cc-40a1-8fd3-2760a0c45887 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:51.287357 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:51.293163 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:51.303135 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36215a-4465-4b93-ae53-8dcfc57683d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:51.304044 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36215a-4465-4b93-ae53-8dcfc57683d8 None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:51.304477 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36215a-4465-4b93-ae53-8dcfc57683d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:51.304971 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36215a-4465-4b93-ae53-8dcfc57683d8 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:51.306719 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb36215a-4465-4b93-ae53-8dcfc57683d8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:51.308328 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb36215a-4465-4b93-ae53-8dcfc57683d8 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:51.327968 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 359/728] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Jun 15 14:29:51 2022] POST /identity/v3/services => generated 240 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:29:51.329338 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78921e12-20f2-44b8-8de7-2e8128fe0446 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:51.329635 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78921e12-20f2-44b8-8de7-2e8128fe0446 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:51.329917 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78921e12-20f2-44b8-8de7-2e8128fe0446 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:51.331461 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 360/729] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:51.333384 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce5909-36fc-47f2-ad17-2283b3ce3c13 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:51.334132 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce5909-36fc-47f2-ad17-2283b3ce3c13 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:51.334570 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce5909-36fc-47f2-ad17-2283b3ce3c13 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:51.336821 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 361/730] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:51.344151 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a846809-94d4-4e27-9bbc-065534c5ead0 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:51.344438 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a846809-94d4-4e27-9bbc-065534c5ead0 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:51.344724 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a846809-94d4-4e27-9bbc-065534c5ead0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:51.370393 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-3a846809-94d4-4e27-9bbc-065534c5ead0 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:51.372568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-3a846809-94d4-4e27-9bbc-065534c5ead0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:51.412816 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ff8c9-b3cc-40a1-8fd3-2760a0c45887 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:51.462437 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb371f3-e3d7-4a4a-834f-12dc150ec18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:51.463479 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb371f3-e3d7-4a4a-834f-12dc150ec18e None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:51.463952 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb371f3-e3d7-4a4a-834f-12dc150ec18e None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:51.464544 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb371f3-e3d7-4a4a-834f-12dc150ec18e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:51.467131 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 362/731] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:51 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:51.468887 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa41a8-788e-4b2c-b96d-9954dce39fcd None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:51.469378 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa41a8-788e-4b2c-b96d-9954dce39fcd None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:51.469853 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa41a8-788e-4b2c-b96d-9954dce39fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:51.480512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb371f3-e3d7-4a4a-834f-12dc150ec18e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:51.482534 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb371f3-e3d7-4a4a-834f-12dc150ec18e None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:51.496897 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-31aa41a8-788e-4b2c-b96d-9954dce39fcd None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:51.498032 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-31aa41a8-788e-4b2c-b96d-9954dce39fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:51.499884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-4eb371f3-e3d7-4a4a-834f-12dc150ec18e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:29:51.499884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:51.499884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:51.499884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:51.499884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:51.499884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:51.499884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:51.499884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:51.499884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:51.499884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:51.499884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:51.499884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:51.499884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:29:51.499884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:29:51.499884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:29:51.499884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:29:51.500809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:51.500809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:51.500809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:29:51.500809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:51.500809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:29:51.500809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:51.500809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:51.500809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:51.500809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:51.500809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:51.500809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:51.500809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:51.500809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:51.500809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:51.500809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:51.500809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:51.500809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:51.500809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:51.501724 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:51.501724 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:51.501724 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:29:51.501724 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:29:51.501724 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:29:51.501724 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:29:51.501724 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:29:51.501724 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:29:51.501724 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:29:51.501724 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:51.504117 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 370/732] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:29:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:51.509849 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb371f3-e3d7-4a4a-834f-12dc150ec18e None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:51.582364 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:51.603787 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 363/733] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:51 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:51.605551 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa41a8-788e-4b2c-b96d-9954dce39fcd None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:51.627882 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb371f3-e3d7-4a4a-834f-12dc150ec18e None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:51.667312 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e12eb1-341a-4435-b1b4-25c59dea14d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:51.668792 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:51.669518 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e12eb1-341a-4435-b1b4-25c59dea14d3 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:51.669811 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e12eb1-341a-4435-b1b4-25c59dea14d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:51.670149 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e12eb1-341a-4435-b1b4-25c59dea14d3 None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:51.671093 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e12eb1-341a-4435-b1b4-25c59dea14d3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:51.671456 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e12eb1-341a-4435-b1b4-25c59dea14d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:51.672910 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e12eb1-341a-4435-b1b4-25c59dea14d3 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:51.695297 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 371/734] 10.222.0.45 () {56 vars in 1135 bytes} [Wed Jun 15 14:29:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:51.700126 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e12eb1-341a-4435-b1b4-25c59dea14d3 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:51.741358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa41a8-788e-4b2c-b96d-9954dce39fcd None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:51.783236 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee648a-d345-4fbb-8953-6b059a044d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:51.784535 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee648a-d345-4fbb-8953-6b059a044d66 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:51.784827 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee648a-d345-4fbb-8953-6b059a044d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:51.785104 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee648a-d345-4fbb-8953-6b059a044d66 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:51.799504 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee648a-d345-4fbb-8953-6b059a044d66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:51.809634 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-7eee648a-d345-4fbb-8953-6b059a044d66 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:51.809634 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:51.809634 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:51.809634 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:51.809634 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:51.809634 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:51.809634 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:51.809634 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:51.809634 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:51.809634 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:51.809634 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:51.809634 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:51.809634 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:51.809634 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:51.809634 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:51.809634 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:51.810601 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:51.810601 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:51.810601 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:51.810601 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:51.810601 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:51.810601 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:51.810601 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:51.810601 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:51.810601 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:51.810601 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:51.810601 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:51.810601 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:51.810601 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:51.810601 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:51.810601 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:51.810601 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:51.811448 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:51.811448 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:51.811448 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:51.811448 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:51.811448 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:51.811448 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:51.811448 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:51.811448 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:51.811448 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:51.811448 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:51.811448 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:51.811448 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:51.812100 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 364/735] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Jun 15 14:29:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:51.815054 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee648a-d345-4fbb-8953-6b059a044d66 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:51.823613 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e12eb1-341a-4435-b1b4-25c59dea14d3 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:51.844064 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:51.871213 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d403f1-329d-4e8f-bc92-66688fd6582c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:51.872412 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d403f1-329d-4e8f-bc92-66688fd6582c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:51.872703 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d403f1-329d-4e8f-bc92-66688fd6582c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:51.872976 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d403f1-329d-4e8f-bc92-66688fd6582c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:51.889037 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d403f1-329d-4e8f-bc92-66688fd6582c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:51.898767 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-b3d403f1-329d-4e8f-bc92-66688fd6582c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:29:51.898767 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:51.898767 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:51.898767 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:51.898767 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:51.898767 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:51.898767 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:51.898767 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:51.898767 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:51.898767 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:51.898767 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:51.898767 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:51.898767 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:51.898767 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:51.898767 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:51.898767 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:51.900131 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:51.900131 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:51.900131 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:51.900131 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:51.900131 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:51.900131 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:51.900131 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:51.900131 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:51.900131 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:51.900131 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:51.900131 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:51.900131 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:51.900131 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:51.900131 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:51.900131 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:51.900131 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:51.902545 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:51.902545 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:51.902545 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:51.902545 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:51.902545 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:51.902545 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:51.902545 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:51.902545 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:51.902545 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:51.902545 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:51.902545 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:29:51.902545 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:51.902545 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 372/736] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:29:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:51.902545 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d403f1-329d-4e8f-bc92-66688fd6582c None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:51.940990 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee648a-d345-4fbb-8953-6b059a044d66 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:51.949534 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:51.980863 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c0d83f-ce17-4731-8b2a-90abdd1a00cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:51.981621 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c0d83f-ce17-4731-8b2a-90abdd1a00cb None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:51.981907 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c0d83f-ce17-4731-8b2a-90abdd1a00cb None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:51.982179 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c0d83f-ce17-4731-8b2a-90abdd1a00cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:52.017751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d403f1-329d-4e8f-bc92-66688fd6582c None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:52.055460 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c0d83f-ce17-4731-8b2a-90abdd1a00cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:52.056459 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ae2492-30ef-47ff-a54e-ee98bddc4aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:52.056938 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c0d83f-ce17-4731-8b2a-90abdd1a00cb None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:52.057127 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae2492-30ef-47ff-a54e-ee98bddc4aed None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:52.057408 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae2492-30ef-47ff-a54e-ee98bddc4aed None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:52.057680 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae2492-30ef-47ff-a54e-ee98bddc4aed None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:52.058949 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ae2492-30ef-47ff-a54e-ee98bddc4aed None admin] RBAC: Adding query filter params (name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:52.059293 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ae2492-30ef-47ff-a54e-ee98bddc4aed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:52.060649 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ae2492-30ef-47ff-a54e-ee98bddc4aed None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:52.073122 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 373/737] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Jun 15 14:29:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:52.073122 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ae2492-30ef-47ff-a54e-ee98bddc4aed None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:52.120154 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:52.171012 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 365/738] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:51 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:52.172436 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c9ce8-61b5-4c8a-88b6-1051d112e7b0 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:52.172782 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c9ce8-61b5-4c8a-88b6-1051d112e7b0 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:52.173125 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c9ce8-61b5-4c8a-88b6-1051d112e7b0 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:52.174557 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 366/739] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:52.175096 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c9ce8-61b5-4c8a-88b6-1051d112e7b0 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:52.186489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:52.195046 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ae2492-30ef-47ff-a54e-ee98bddc4aed None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:52.230028 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a96e8-b9f2-4e25-a94f-5a2a56d55ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:52.230695 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a96e8-b9f2-4e25-a94f-5a2a56d55ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:52.231157 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a96e8-b9f2-4e25-a94f-5a2a56d55ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:52.231595 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a96e8-b9f2-4e25-a94f-5a2a56d55ee9 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:52.232345 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355a96e8-b9f2-4e25-a94f-5a2a56d55ee9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:52.232653 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355a96e8-b9f2-4e25-a94f-5a2a56d55ee9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:52.234012 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355a96e8-b9f2-4e25-a94f-5a2a56d55ee9 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:52.242111 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 374/740] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:52.243094 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a96e8-b9f2-4e25-a94f-5a2a56d55ee9 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:52.300141 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c9ce8-61b5-4c8a-88b6-1051d112e7b0 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:52.361476 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00becb37-9a3f-4270-8dea-96fb80e7da97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:52.362194 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00becb37-9a3f-4270-8dea-96fb80e7da97 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:52.362471 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00becb37-9a3f-4270-8dea-96fb80e7da97 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:52.362741 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00becb37-9a3f-4270-8dea-96fb80e7da97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:52.376387 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00becb37-9a3f-4270-8dea-96fb80e7da97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:52.378048 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00becb37-9a3f-4270-8dea-96fb80e7da97 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:52.393898 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a96e8-b9f2-4e25-a94f-5a2a56d55ee9 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:52.395801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:52.398482 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-00becb37-9a3f-4270-8dea-96fb80e7da97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:52.398482 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:52.398482 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:52.398482 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:52.398482 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:52.398482 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:52.398482 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:52.398482 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:52.398482 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:52.398482 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:52.398482 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:52.398482 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:52.398482 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:52.398482 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:52.398482 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:52.399403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:52.399403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:52.399403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:52.399403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:52.399403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:52.399403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:52.399403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:52.399403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:52.399403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:52.399403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:52.399403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:52.399403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:52.399403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:52.399403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:52.399403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:52.399403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:52.399403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:52.400332 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:52.400332 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:52.400332 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:52.400332 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:52.404675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 367/741] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:52.405999 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb34a8b-dbc9-46f6-b0e8-fe078dc6b519 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:52.406459 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb34a8b-dbc9-46f6-b0e8-fe078dc6b519 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:52.406878 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb34a8b-dbc9-46f6-b0e8-fe078dc6b519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:52.424053 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-0cb34a8b-dbc9-46f6-b0e8-fe078dc6b519 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:52.425431 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-0cb34a8b-dbc9-46f6-b0e8-fe078dc6b519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:52.436751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b004e17-2836-485d-8df9-b99ea34d4f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:52.437954 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b004e17-2836-485d-8df9-b99ea34d4f2c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:52.438247 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b004e17-2836-485d-8df9-b99ea34d4f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:52.438524 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b004e17-2836-485d-8df9-b99ea34d4f2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:52.453280 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b004e17-2836-485d-8df9-b99ea34d4f2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:52.455361 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b004e17-2836-485d-8df9-b99ea34d4f2c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:52.482351 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-7b004e17-2836-485d-8df9-b99ea34d4f2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:52.482351 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:52.482351 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:52.482351 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:52.482351 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:52.482351 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:52.482351 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:52.482351 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:52.482351 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:52.482351 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:52.482351 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:52.482351 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:52.482351 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:52.482351 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:52.482351 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:52.483490 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:52.483490 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:52.483490 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:52.483490 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:52.483490 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:52.483490 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:52.483490 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:52.483490 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:52.483490 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:52.483490 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:52.483490 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:52.483490 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:52.483490 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:52.483490 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:52.483490 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:52.483490 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:52.483490 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:52.485494 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:52.485494 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:52.485494 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:52.485494 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:52.485494 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 375/742] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:52.485494 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b004e17-2836-485d-8df9-b99ea34d4f2c None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:52.489631 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:52.506820 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 368/743] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:52 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:52.508067 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb34a8b-dbc9-46f6-b0e8-fe078dc6b519 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:52.614601 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b004e17-2836-485d-8df9-b99ea34d4f2c None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:52.644495 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb34a8b-dbc9-46f6-b0e8-fe078dc6b519 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:52.662596 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd5218f-11d2-4e78-be31-ec5f0660d0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:52.663429 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5218f-11d2-4e78-be31-ec5f0660d0c4 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:52.663967 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5218f-11d2-4e78-be31-ec5f0660d0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:52.663967 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5218f-11d2-4e78-be31-ec5f0660d0c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:52.675220 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd5218f-11d2-4e78-be31-ec5f0660d0c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:52.676656 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd5218f-11d2-4e78-be31-ec5f0660d0c4 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:52.685301 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b10dd4b-3d38-49e2-89b6-ed7241b8f4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:52.685975 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10dd4b-3d38-49e2-89b6-ed7241b8f4af None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:52.686261 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10dd4b-3d38-49e2-89b6-ed7241b8f4af None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:52.686544 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10dd4b-3d38-49e2-89b6-ed7241b8f4af None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:52.687304 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b10dd4b-3d38-49e2-89b6-ed7241b8f4af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:52.687618 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b10dd4b-3d38-49e2-89b6-ed7241b8f4af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:52.688976 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b10dd4b-3d38-49e2-89b6-ed7241b8f4af None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:52.694100 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-9fd5218f-11d2-4e78-be31-ec5f0660d0c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:52.694100 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:52.694100 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:52.694100 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:52.694100 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:52.694100 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:52.694100 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:52.694100 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:52.694100 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:52.694100 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:52.694100 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:52.694100 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:52.694100 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:52.694100 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:52.694100 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:52.694951 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:52.694951 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:52.694951 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:52.694951 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:52.694951 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:52.694951 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:52.694951 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:52.694951 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:52.694951 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:52.694951 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:52.694951 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:52.694951 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:52.694951 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:52.694951 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:52.694951 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:52.694951 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:52.694951 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:52.700845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:52.700845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:52.700845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:52.700845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:52.700845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 376/744] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:52.700845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd5218f-11d2-4e78-be31-ec5f0660d0c4 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:52.710319 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:52.712282 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 369/745] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:52.712949 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b10dd4b-3d38-49e2-89b6-ed7241b8f4af None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:52.763642 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:52.814976 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd5218f-11d2-4e78-be31-ec5f0660d0c4 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:52.816611 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b10dd4b-3d38-49e2-89b6-ed7241b8f4af None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:52.849857 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ef2a47-1fbe-4214-be97-50f99ea2453e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:52.850053 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f7c29-c3dc-46d3-b91a-9a3b91129022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:52.850518 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef2a47-1fbe-4214-be97-50f99ea2453e None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:52.850791 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f7c29-c3dc-46d3-b91a-9a3b91129022 None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:52.850899 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef2a47-1fbe-4214-be97-50f99ea2453e None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:52.851199 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef2a47-1fbe-4214-be97-50f99ea2453e None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:52.851318 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f7c29-c3dc-46d3-b91a-9a3b91129022 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:52.851392 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f7c29-c3dc-46d3-b91a-9a3b91129022 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:52.851822 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ef2a47-1fbe-4214-be97-50f99ea2453e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:52.852084 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7f7c29-c3dc-46d3-b91a-9a3b91129022 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:52.852178 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ef2a47-1fbe-4214-be97-50f99ea2453e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:52.853468 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ef2a47-1fbe-4214-be97-50f99ea2453e None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:52.853567 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7f7c29-c3dc-46d3-b91a-9a3b91129022 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:52.872539 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 377/746] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:52.872539 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ef2a47-1fbe-4214-be97-50f99ea2453e None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:52.879384 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 370/747] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Jun 15 14:29:52 2022] POST /identity/v3/services => generated 251 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:29:52.880218 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f7c29-c3dc-46d3-b91a-9a3b91129022 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:52.969648 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:53.018503 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ef2a47-1fbe-4214-be97-50f99ea2453e None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:53.045059 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f7c29-c3dc-46d3-b91a-9a3b91129022 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:53.048427 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:53.068092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a24fa-819b-4058-b1c4-308d2c15bc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:53.069112 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a24fa-819b-4058-b1c4-308d2c15bc10 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:53.069112 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a24fa-819b-4058-b1c4-308d2c15bc10 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:53.069616 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a24fa-819b-4058-b1c4-308d2c15bc10 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:53.070028 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a24fa-819b-4058-b1c4-308d2c15bc10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:53.070337 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a24fa-819b-4058-b1c4-308d2c15bc10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:53.071858 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a24fa-819b-4058-b1c4-308d2c15bc10 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:53.087621 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 378/748] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:53.088734 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a24fa-819b-4058-b1c4-308d2c15bc10 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:53.098158 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b8b58b-61ce-4315-a6d4-c179791659be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:53.098973 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8b58b-61ce-4315-a6d4-c179791659be None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:53.099294 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8b58b-61ce-4315-a6d4-c179791659be None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:53.099570 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8b58b-61ce-4315-a6d4-c179791659be None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:53.108815 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b8b58b-61ce-4315-a6d4-c179791659be None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:53.110290 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b8b58b-61ce-4315-a6d4-c179791659be None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:53.128091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-89b8b58b-61ce-4315-a6d4-c179791659be None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:29:53.128091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:53.128091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:53.128091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:53.128091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:53.128091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:53.128091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:53.128091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:53.128091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:53.128091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:53.128091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:53.128091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:53.128091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:29:53.128091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:29:53.128091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:29:53.128091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:29:53.129092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:53.129092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:53.129092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:29:53.129092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:53.129092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:29:53.129092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:53.129092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:53.129092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:53.129092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:53.129092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:53.129092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:53.129092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:53.129092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:53.129092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:53.129092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:53.129092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:53.129092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:53.129092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:53.130026 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:53.130026 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:53.130026 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:29:53.130026 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:29:53.130026 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:29:53.130026 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:29:53.130026 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:29:53.130026 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:29:53.130026 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:29:53.130026 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:53.130026 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 371/749] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Jun 15 14:29:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:53.130597 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b8b58b-61ce-4315-a6d4-c179791659be None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:53.197069 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a24fa-819b-4058-b1c4-308d2c15bc10 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:53.227842 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:53.234297 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319baf2-e386-4158-bfb6-4dc717416980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:53.235359 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319baf2-e386-4158-bfb6-4dc717416980 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:53.235646 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319baf2-e386-4158-bfb6-4dc717416980 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:53.236299 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319baf2-e386-4158-bfb6-4dc717416980 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:53.245692 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6319baf2-e386-4158-bfb6-4dc717416980 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:53.247874 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6319baf2-e386-4158-bfb6-4dc717416980 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:53.252641 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b8b58b-61ce-4315-a6d4-c179791659be None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:53.266565 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-6319baf2-e386-4158-bfb6-4dc717416980 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:53.266565 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:53.266565 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:53.266565 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:53.266565 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:53.266565 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:53.266565 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:53.266565 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:53.266565 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:53.266565 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:53.266565 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:53.266565 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:53.266565 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:29:53.266565 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:29:53.266565 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:29:53.267512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:29:53.267512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:53.267512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:53.267512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:53.267512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:53.267512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:53.267512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:53.267512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:53.267512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:53.267512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:53.267512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:53.267512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:53.267512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:53.267512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:53.267512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:53.267512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:53.267512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:53.268389 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:29:53.268389 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:29:53.268389 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:29:53.268389 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:29:53.268389 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:29:53.268389 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:29:53.268389 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:53.268389 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:53.268389 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 379/750] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:53.268861 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319baf2-e386-4158-bfb6-4dc717416980 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:53.295928 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20970fdf-52ea-406f-bc17-c48126843bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:53.297277 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20970fdf-52ea-406f-bc17-c48126843bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:53.297578 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20970fdf-52ea-406f-bc17-c48126843bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:53.297860 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20970fdf-52ea-406f-bc17-c48126843bd8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:53.309140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20970fdf-52ea-406f-bc17-c48126843bd8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:53.310846 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20970fdf-52ea-406f-bc17-c48126843bd8 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:53.330088 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-20970fdf-52ea-406f-bc17-c48126843bd8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:29:53.330088 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:53.330088 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:53.330088 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:53.330088 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:53.330088 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:53.330088 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:53.330088 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:53.330088 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:53.330088 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:53.330088 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:53.330088 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:53.330088 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:29:53.330088 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:29:53.330088 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:29:53.330088 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:29:53.332264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:53.332264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:53.332264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:29:53.332264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:53.332264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:29:53.332264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:53.332264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:53.332264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:53.332264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:53.332264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:53.332264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:53.332264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:53.332264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:53.332264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:53.332264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:53.332264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:53.332264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:53.332264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:53.334487 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:53.334487 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:53.334487 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:29:53.334487 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:29:53.334487 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:29:53.334487 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:29:53.334487 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:29:53.334487 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:29:53.334487 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:29:53.334487 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:53.335823 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 372/751] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Jun 15 14:29:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:53.335975 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20970fdf-52ea-406f-bc17-c48126843bd8 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:53.369951 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:53.398028 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319baf2-e386-4158-bfb6-4dc717416980 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:53.438519 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88944117-d787-495d-a358-ca9432e1c823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:53.439676 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88944117-d787-495d-a358-ca9432e1c823 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:53.440075 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88944117-d787-495d-a358-ca9432e1c823 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:53.440414 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88944117-d787-495d-a358-ca9432e1c823 None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:53.441605 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88944117-d787-495d-a358-ca9432e1c823 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:53.441917 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88944117-d787-495d-a358-ca9432e1c823 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:53.443674 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88944117-d787-495d-a358-ca9432e1c823 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:53.463159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 380/752] 10.222.0.45 () {56 vars in 1149 bytes} [Wed Jun 15 14:29:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:53.469002 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88944117-d787-495d-a358-ca9432e1c823 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:53.471099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20970fdf-52ea-406f-bc17-c48126843bd8 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:53.517095 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027f66f-3086-49a3-926a-faf4dae01274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:53.517929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027f66f-3086-49a3-926a-faf4dae01274 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:53.518282 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027f66f-3086-49a3-926a-faf4dae01274 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:53.518626 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027f66f-3086-49a3-926a-faf4dae01274 None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:53.519548 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3027f66f-3086-49a3-926a-faf4dae01274 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:53.520180 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3027f66f-3086-49a3-926a-faf4dae01274 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:53.521707 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3027f66f-3086-49a3-926a-faf4dae01274 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:53.531318 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:53.536527 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 373/753] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:29:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:53.537576 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027f66f-3086-49a3-926a-faf4dae01274 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:53.597066 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88944117-d787-495d-a358-ca9432e1c823 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:53.631498 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230b7c8-8303-47c2-bc85-ffc8ad716e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:53.632174 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230b7c8-8303-47c2-bc85-ffc8ad716e69 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:53.632695 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230b7c8-8303-47c2-bc85-ffc8ad716e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:53.632809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230b7c8-8303-47c2-bc85-ffc8ad716e69 None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:53.633557 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230b7c8-8303-47c2-bc85-ffc8ad716e69 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:53.633867 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230b7c8-8303-47c2-bc85-ffc8ad716e69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:53.635326 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230b7c8-8303-47c2-bc85-ffc8ad716e69 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:53.640555 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:53.657713 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 381/754] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Jun 15 14:29:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:53.657713 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230b7c8-8303-47c2-bc85-ffc8ad716e69 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:53.665019 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027f66f-3086-49a3-926a-faf4dae01274 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:53.709600 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8fbe0a-538a-4654-82b5-605ef9abb3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:53.710283 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8fbe0a-538a-4654-82b5-605ef9abb3e6 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:53.710565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8fbe0a-538a-4654-82b5-605ef9abb3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:53.710838 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8fbe0a-538a-4654-82b5-605ef9abb3e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:53.760941 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230b7c8-8303-47c2-bc85-ffc8ad716e69 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:53.763547 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:53.786405 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8fbe0a-538a-4654-82b5-605ef9abb3e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:53.787884 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8fbe0a-538a-4654-82b5-605ef9abb3e6 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:53.799087 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167bdff1-6c48-47a7-b198-e52be83c2056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:53.799931 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167bdff1-6c48-47a7-b198-e52be83c2056 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:53.800354 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167bdff1-6c48-47a7-b198-e52be83c2056 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:53.800769 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167bdff1-6c48-47a7-b198-e52be83c2056 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:53.801916 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167bdff1-6c48-47a7-b198-e52be83c2056 None admin] RBAC: Adding query filter params (role.id=ed1cfa2fcf724a4d84f5acea100930c7, scope.project.id=d488c79cb1704e9496fc196d8aa6c104, user.id=641a81b3140f47119b5c2323a97a6681) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:53.802228 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167bdff1-6c48-47a7-b198-e52be83c2056 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:53.803673 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167bdff1-6c48-47a7-b198-e52be83c2056 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:53.819426 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 382/755] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:29:53 2022] GET /identity/v3/role_assignments?user.id=641a81b3140f47119b5c2323a97a6681&scope.project.id=d488c79cb1704e9496fc196d8aa6c104&role.id=ed1cfa2fcf724a4d84f5acea100930c7 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:53.821718 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4a6e6-911d-4c7b-921a-62dcf882d714 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:53.822015 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4a6e6-911d-4c7b-921a-62dcf882d714 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:53.822298 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4a6e6-911d-4c7b-921a-62dcf882d714 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:53.824730 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 383/756] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:53.825539 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4a6e6-911d-4c7b-921a-62dcf882d714 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:53.880252 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:53.905506 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 374/757] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:53 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:29:53.907374 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b5713-2964-4ab5-a718-b72f342292cc None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:53.907963 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b5713-2964-4ab5-a718-b72f342292cc None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:53.908437 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b5713-2964-4ab5-a718-b72f342292cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:53.927627 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-7f9b5713-2964-4ab5-a718-b72f342292cc None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:53.929206 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-7f9b5713-2964-4ab5-a718-b72f342292cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:53.947229 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4a6e6-911d-4c7b-921a-62dcf882d714 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:53.987082 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a67cb8-78dd-4810-8795-35b533d7bcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:53.987886 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a67cb8-78dd-4810-8795-35b533d7bcb6 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:53.988172 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a67cb8-78dd-4810-8795-35b533d7bcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:53.988448 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a67cb8-78dd-4810-8795-35b533d7bcb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:54.000866 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a67cb8-78dd-4810-8795-35b533d7bcb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:54.001490 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a67cb8-78dd-4810-8795-35b533d7bcb6 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:54.009754 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 375/758] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:29:54.010648 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b5713-2964-4ab5-a718-b72f342292cc None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:54.036417 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-64a67cb8-78dd-4810-8795-35b533d7bcb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:54.036417 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:54.036417 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:54.036417 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:54.036417 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:54.036417 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:54.036417 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:54.036417 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:54.036417 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:54.036417 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:54.036417 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:54.036417 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:54.036417 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:54.036417 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:54.036417 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:54.037362 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:54.037362 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:54.037362 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:54.037362 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:54.037362 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:54.037362 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:54.037362 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:54.037362 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:54.037362 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:54.037362 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:54.037362 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:54.037362 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:54.037362 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:54.037362 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:54.037362 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:54.037362 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:54.037362 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:54.038254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:54.038254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:54.038254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:54.038254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:54.038254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 384/759] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:54.038960 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a67cb8-78dd-4810-8795-35b533d7bcb6 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:54.072821 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:54.132146 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b5713-2964-4ab5-a718-b72f342292cc None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:54.151400 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:54.175570 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651655f-d6fe-40d3-b6dc-8bae59a8dc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:54.176786 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651655f-d6fe-40d3-b6dc-8bae59a8dc51 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:54.177263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651655f-d6fe-40d3-b6dc-8bae59a8dc51 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:54.177756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651655f-d6fe-40d3-b6dc-8bae59a8dc51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:54.186611 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a67cb8-78dd-4810-8795-35b533d7bcb6 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:54.191060 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a651655f-d6fe-40d3-b6dc-8bae59a8dc51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:54.192724 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a651655f-d6fe-40d3-b6dc-8bae59a8dc51 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:54.212662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-a651655f-d6fe-40d3-b6dc-8bae59a8dc51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:54.212662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:54.212662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:54.212662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:54.212662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:54.212662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:54.212662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:54.212662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:54.212662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:54.212662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:54.212662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:54.212662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:54.212662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:54.212662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:54.212662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:54.213626 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:54.213626 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:54.213626 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:54.213626 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:54.213626 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:54.213626 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:54.213626 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:54.213626 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:54.213626 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:54.213626 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:54.213626 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:54.213626 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:54.213626 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:54.213626 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:54.213626 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:54.213626 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:54.213626 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:54.214536 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:54.214536 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:54.214536 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:54.214536 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:54.216828 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 376/760] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:54.217864 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651655f-d6fe-40d3-b6dc-8bae59a8dc51 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:54.225491 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ac4af-0836-447e-9d12-9de444b3850a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:54.226586 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ac4af-0836-447e-9d12-9de444b3850a None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:54.227108 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ac4af-0836-447e-9d12-9de444b3850a None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:54.227570 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ac4af-0836-447e-9d12-9de444b3850a None admin] PATH_INFO: `/v3/services/network` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:54.228412 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68ac4af-0836-447e-9d12-9de444b3850a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:54.229968 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68ac4af-0836-447e-9d12-9de444b3850a None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:54.243176 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-a68ac4af-0836-447e-9d12-9de444b3850a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:29:54.243176 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:54.243176 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:29:54.243176 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:29:54.243176 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:29:54.243176 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:29:54.243176 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:29:54.243176 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:29:54.243176 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:29:54.243176 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:54.243176 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:29:54.243176 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:54.243176 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:54.243176 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:54.243176 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:54.243176 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:54.243176 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:54.244564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:54.244564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:54.244564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:54.244564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:54.244564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:54.244564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:54.244564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:29:54.244564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:29:54.244564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:29:54.244564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:29:54.244564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:54.244564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:54.244564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:54.244564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:54.244564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:54.244564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:54.245401 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:54.245401 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:54.245401 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:54.245401 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:54.245401 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:54.245401 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:54.245401 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:54.245401 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:54.245401 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:54.245401 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:54.245401 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:29:54.245401 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:29:54.245401 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:29:54.245401 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:54.248924 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 385/761] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:54 2022] GET /identity/v3/services/network => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:54.249999 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ac4af-0836-447e-9d12-9de444b3850a None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:54.326652 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651655f-d6fe-40d3-b6dc-8bae59a8dc51 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:54.348839 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:54.367964 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c292bfd-cbfa-4f9d-86c2-7f0a89585fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:54.369058 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c292bfd-cbfa-4f9d-86c2-7f0a89585fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:54.369561 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c292bfd-cbfa-4f9d-86c2-7f0a89585fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:54.369930 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c292bfd-cbfa-4f9d-86c2-7f0a89585fe2 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:54.370902 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c292bfd-cbfa-4f9d-86c2-7f0a89585fe2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:54.371334 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c292bfd-cbfa-4f9d-86c2-7f0a89585fe2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:54.372065 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ac4af-0836-447e-9d12-9de444b3850a None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:54.372928 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c292bfd-cbfa-4f9d-86c2-7f0a89585fe2 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:54.386227 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 377/762] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:54.387069 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c292bfd-cbfa-4f9d-86c2-7f0a89585fe2 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:54.418395 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:54.420377 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3af4b-8b3f-4f54-b134-433fb919900b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:54.421481 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3af4b-8b3f-4f54-b134-433fb919900b None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:54.421910 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3af4b-8b3f-4f54-b134-433fb919900b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:54.422331 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3af4b-8b3f-4f54-b134-433fb919900b None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:54.423383 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa3af4b-8b3f-4f54-b134-433fb919900b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:54.424792 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa3af4b-8b3f-4f54-b134-433fb919900b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:54.426660 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa3af4b-8b3f-4f54-b134-433fb919900b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:54.448252 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 386/763] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:54.448252 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3af4b-8b3f-4f54-b134-433fb919900b None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:54.519621 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c292bfd-cbfa-4f9d-86c2-7f0a89585fe2 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:54.559233 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a10d27-f823-434e-87ad-c1c39e4ed905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:54.560261 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a10d27-f823-434e-87ad-c1c39e4ed905 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:54.560762 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a10d27-f823-434e-87ad-c1c39e4ed905 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:54.561251 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a10d27-f823-434e-87ad-c1c39e4ed905 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:54.562272 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a10d27-f823-434e-87ad-c1c39e4ed905 None admin] RBAC: Adding query filter params (name=network) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:54.562801 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a10d27-f823-434e-87ad-c1c39e4ed905 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:54.564420 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a10d27-f823-434e-87ad-c1c39e4ed905 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:54.569896 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 378/764] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Jun 15 14:29:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:54.571030 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a10d27-f823-434e-87ad-c1c39e4ed905 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:54.594223 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3af4b-8b3f-4f54-b134-433fb919900b None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:54.631331 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f5428-b524-4804-b193-ba7ca1ed850d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:54.632157 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f5428-b524-4804-b193-ba7ca1ed850d None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:54.632851 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f5428-b524-4804-b193-ba7ca1ed850d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:54.633128 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f5428-b524-4804-b193-ba7ca1ed850d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:54.643146 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f5428-b524-4804-b193-ba7ca1ed850d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:54.645153 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f5428-b524-4804-b193-ba7ca1ed850d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:54.663159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-189f5428-b524-4804-b193-ba7ca1ed850d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:54.663159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:54.663159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:54.663159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:54.663159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:54.663159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:54.663159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:54.663159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:54.663159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:54.663159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:54.663159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:54.663159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:54.663159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:29:54.663159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:29:54.663159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:29:54.664096 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:29:54.664096 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:54.664096 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:54.664096 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:54.664096 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:54.664096 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:54.664096 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:54.664096 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:54.664096 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:54.664096 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:54.664096 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:54.664096 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:54.664096 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:54.664096 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:54.664096 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:54.664096 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:54.664096 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:54.664960 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:29:54.664960 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:29:54.664960 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:29:54.664960 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:29:54.664960 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:29:54.664960 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:29:54.664960 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:54.664960 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:54.664960 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 387/765] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:54.665968 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f5428-b524-4804-b193-ba7ca1ed850d None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:54.669162 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:54.679545 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:54.691841 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a10d27-f823-434e-87ad-c1c39e4ed905 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:54.742360 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5776bbc3-974a-4b49-aa68-c6bb291f749e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:54.743631 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5776bbc3-974a-4b49-aa68-c6bb291f749e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:54.744475 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5776bbc3-974a-4b49-aa68-c6bb291f749e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:54.744912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5776bbc3-974a-4b49-aa68-c6bb291f749e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:54.754296 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5776bbc3-974a-4b49-aa68-c6bb291f749e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:54.757041 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5776bbc3-974a-4b49-aa68-c6bb291f749e None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:54.775718 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-5776bbc3-974a-4b49-aa68-c6bb291f749e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:54.775718 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:54.775718 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:54.775718 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:54.775718 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:54.775718 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:54.775718 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:54.775718 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:54.775718 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:54.775718 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:54.775718 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:54.775718 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:54.775718 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:29:54.775718 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:29:54.775718 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:29:54.776970 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:29:54.776970 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:54.776970 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:54.776970 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:54.776970 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:54.776970 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:54.776970 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:54.776970 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:54.776970 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:54.776970 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:54.776970 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:54.776970 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:54.776970 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:54.776970 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:54.776970 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:54.776970 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:54.776970 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:54.778559 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:29:54.778559 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:29:54.778559 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:29:54.778559 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:29:54.778559 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:29:54.778559 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:29:54.778559 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:54.778559 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:54.781549 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 379/766] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:54.782484 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5776bbc3-974a-4b49-aa68-c6bb291f749e None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:54.795372 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f5428-b524-4804-b193-ba7ca1ed850d None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:54.833404 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44051cf1-3264-49a5-bc17-8afed0fc4f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:54.834227 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44051cf1-3264-49a5-bc17-8afed0fc4f14 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:54.834609 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44051cf1-3264-49a5-bc17-8afed0fc4f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:54.835058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44051cf1-3264-49a5-bc17-8afed0fc4f14 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:54.835892 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44051cf1-3264-49a5-bc17-8afed0fc4f14 None admin] RBAC: Adding query filter params (type=network) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:54.836332 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44051cf1-3264-49a5-bc17-8afed0fc4f14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:54.837806 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44051cf1-3264-49a5-bc17-8afed0fc4f14 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:54.843513 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 388/767] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Jun 15 14:29:54 2022] GET /identity/v3/services?type=network => generated 350 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:54.844578 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44051cf1-3264-49a5-bc17-8afed0fc4f14 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:54.914805 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:54.919674 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5776bbc3-974a-4b49-aa68-c6bb291f749e None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:54.957612 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44051cf1-3264-49a5-bc17-8afed0fc4f14 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:54.966554 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14772f2a-b4ea-4277-a2b5-4e896564e804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:54.967473 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14772f2a-b4ea-4277-a2b5-4e896564e804 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:54.968327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14772f2a-b4ea-4277-a2b5-4e896564e804 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:54.968746 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14772f2a-b4ea-4277-a2b5-4e896564e804 None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:54.970074 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14772f2a-b4ea-4277-a2b5-4e896564e804 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:54.970794 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14772f2a-b4ea-4277-a2b5-4e896564e804 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:54.972643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14772f2a-b4ea-4277-a2b5-4e896564e804 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:54.990411 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 380/768] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:29:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:54.991815 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14772f2a-b4ea-4277-a2b5-4e896564e804 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:54.996691 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a6af58-2461-42e1-a29a-a61702c3f680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:54.996996 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6af58-2461-42e1-a29a-a61702c3f680 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:54.997374 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6af58-2461-42e1-a29a-a61702c3f680 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:54.997747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6af58-2461-42e1-a29a-a61702c3f680 None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:54.998687 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a6af58-2461-42e1-a29a-a61702c3f680 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:54.999125 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a6af58-2461-42e1-a29a-a61702c3f680 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:55.000681 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a6af58-2461-42e1-a29a-a61702c3f680 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:55.007769 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:55.022270 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 389/769] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:29:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:55.023240 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a6af58-2461-42e1-a29a-a61702c3f680 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:55.134419 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14772f2a-b4ea-4277-a2b5-4e896564e804 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:55.140144 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a6af58-2461-42e1-a29a-a61702c3f680 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:55.168692 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:55.177784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d052e5-f68f-4b94-a061-9b8148e8ce07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:55.177966 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc4b1c3-6e21-483b-bdf6-806bdf2939d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:55.178452 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d052e5-f68f-4b94-a061-9b8148e8ce07 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:55.178816 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d052e5-f68f-4b94-a061-9b8148e8ce07 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:55.179106 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc4b1c3-6e21-483b-bdf6-806bdf2939d8 None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:55.179215 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d052e5-f68f-4b94-a061-9b8148e8ce07 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:55.179399 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc4b1c3-6e21-483b-bdf6-806bdf2939d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:55.179680 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc4b1c3-6e21-483b-bdf6-806bdf2939d8 None admin] PATH_INFO: `/v3/projects/d488c79cb1704e9496fc196d8aa6c104/users/f3185bd2458c4e22830975700888e9be/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:55.179942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d052e5-f68f-4b94-a061-9b8148e8ce07 None admin] RBAC: Adding query filter params (interface=public, service_id=4bed4a0ea8eb4f6ea7e12f2ef8d64c66, region_id=RegionOne) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:55.180261 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d052e5-f68f-4b94-a061-9b8148e8ce07 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:55.181631 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d052e5-f68f-4b94-a061-9b8148e8ce07 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:55.181727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc4b1c3-6e21-483b-bdf6-806bdf2939d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=d488c79cb1704e9496fc196d8aa6c104, user_id=f3185bd2458c4e22830975700888e9be, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:55.183107 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc4b1c3-6e21-483b-bdf6-806bdf2939d8 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:55.190196 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 381/770] 10.222.0.45 () {56 vars in 1251 bytes} [Wed Jun 15 14:29:54 2022] GET /identity/v3/endpoints?service_id=4bed4a0ea8eb4f6ea7e12f2ef8d64c66&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:55.190928 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d052e5-f68f-4b94-a061-9b8148e8ce07 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:55.198424 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 390/771] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Jun 15 14:29:55 2022] PUT /identity/v3/projects/d488c79cb1704e9496fc196d8aa6c104/users/f3185bd2458c4e22830975700888e9be/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:29:55.200391 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1fb1fc-9f5e-45ed-a2a7-4362676d9485 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:55.200683 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1fb1fc-9f5e-45ed-a2a7-4362676d9485 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:55.200962 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1fb1fc-9f5e-45ed-a2a7-4362676d9485 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:55.206573 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 391/772] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:55.207547 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1fb1fc-9f5e-45ed-a2a7-4362676d9485 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:55.292580 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:55.311802 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d052e5-f68f-4b94-a061-9b8148e8ce07 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:55.332362 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1fb1fc-9f5e-45ed-a2a7-4362676d9485 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:55.350407 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b4e48-12d3-44e8-b05a-e1452303024b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:55.351740 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b4e48-12d3-44e8-b05a-e1452303024b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:55.352199 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b4e48-12d3-44e8-b05a-e1452303024b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:55.352616 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b4e48-12d3-44e8-b05a-e1452303024b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:55.353649 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b4e48-12d3-44e8-b05a-e1452303024b None admin] RBAC: Adding query filter params (role.id=13dcf68cca9c44888c0fd27a78db3dbe, scope.project.id=d488c79cb1704e9496fc196d8aa6c104, user.id=bf393488726e49338e70017f5cb7c4be) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:55.354245 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b4e48-12d3-44e8-b05a-e1452303024b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:55.356057 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b4e48-12d3-44e8-b05a-e1452303024b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:55.362977 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 382/773] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:29:55 2022] GET /identity/v3/role_assignments?user.id=bf393488726e49338e70017f5cb7c4be&scope.project.id=d488c79cb1704e9496fc196d8aa6c104&role.id=13dcf68cca9c44888c0fd27a78db3dbe => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:55.364351 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbbb56-b062-485d-a442-9cf523f0d31c None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:55.364643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbbb56-b062-485d-a442-9cf523f0d31c None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:55.364919 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbbb56-b062-485d-a442-9cf523f0d31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:55.391596 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d88ba-04ee-486b-bf14-5445982877d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:55.392291 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d88ba-04ee-486b-bf14-5445982877d8 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:55.392572 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d88ba-04ee-486b-bf14-5445982877d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:55.392846 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d88ba-04ee-486b-bf14-5445982877d8 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:55.393558 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6d88ba-04ee-486b-bf14-5445982877d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:55.395296 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6d88ba-04ee-486b-bf14-5445982877d8 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:55.400943 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-37fbbb56-b062-485d-a442-9cf523f0d31c None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:55.403180 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-37fbbb56-b062-485d-a442-9cf523f0d31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:55.409348 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 392/774] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Jun 15 14:29:55 2022] GET /identity/v3/services => generated 772 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:55.488816 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 383/775] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:29:55.514665 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d88ba-04ee-486b-bf14-5445982877d8 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:55.576864 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:55.640163 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d88ba-04ee-486b-bf14-5445982877d8 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:55.680637 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262679c-103a-4a36-861a-3468b594dbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:55.681697 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262679c-103a-4a36-861a-3468b594dbec None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:55.682124 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262679c-103a-4a36-861a-3468b594dbec None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:55.682567 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262679c-103a-4a36-861a-3468b594dbec None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:55.683962 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f262679c-103a-4a36-861a-3468b594dbec None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:55.685551 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f262679c-103a-4a36-861a-3468b594dbec None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:55.698200 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-f262679c-103a-4a36-861a-3468b594dbec None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:29:55.698200 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:55.698200 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:29:55.698200 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:29:55.698200 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:29:55.698200 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:29:55.698200 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:29:55.698200 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:29:55.698200 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:29:55.698200 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:55.698200 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:29:55.698200 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:55.698200 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:55.698200 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:55.698200 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:55.698200 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:55.699448 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:55.699448 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:55.699448 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:55.699448 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:55.699448 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:55.699448 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:55.699448 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:55.699448 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:29:55.699448 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:29:55.699448 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:29:55.699448 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:29:55.699448 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:55.699448 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:55.699448 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:55.699448 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:55.699448 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:55.700464 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:55.700464 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:55.700464 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:55.700464 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:55.700464 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:55.700464 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:55.700464 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:55.700464 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:55.700464 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:55.700464 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:55.700464 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:55.700464 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:29:55.700464 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:29:55.700464 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:29:55.700464 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:55.706238 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 393/776] 10.222.0.45 () {56 vars in 1129 bytes} [Wed Jun 15 14:29:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:55.712213 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fbbb56-b062-485d-a442-9cf523f0d31c None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:55.787268 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:55.826050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fbbb56-b062-485d-a442-9cf523f0d31c None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:55.862919 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f4a0f0-d705-4f5d-b136-d3d8e0e8f091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:55.863696 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4a0f0-d705-4f5d-b136-d3d8e0e8f091 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:55.864091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4a0f0-d705-4f5d-b136-d3d8e0e8f091 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:55.864478 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4a0f0-d705-4f5d-b136-d3d8e0e8f091 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:55.865297 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f4a0f0-d705-4f5d-b136-d3d8e0e8f091 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:55.865719 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f4a0f0-d705-4f5d-b136-d3d8e0e8f091 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:55.867310 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f4a0f0-d705-4f5d-b136-d3d8e0e8f091 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:55.872553 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 384/777] 10.222.0.45 () {56 vars in 1124 bytes} [Wed Jun 15 14:29:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:55.878171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262679c-103a-4a36-861a-3468b594dbec None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:56.009241 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262679c-103a-4a36-861a-3468b594dbec None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:56.024306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:56.046493 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6884d085-60f8-42d1-9c80-e4b018f81ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:56.047197 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884d085-60f8-42d1-9c80-e4b018f81ee0 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:56.047484 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884d085-60f8-42d1-9c80-e4b018f81ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:56.047760 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884d085-60f8-42d1-9c80-e4b018f81ee0 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:56.048474 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6884d085-60f8-42d1-9c80-e4b018f81ee0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:56.048783 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6884d085-60f8-42d1-9c80-e4b018f81ee0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:56.050166 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6884d085-60f8-42d1-9c80-e4b018f81ee0 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:56.055835 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 394/778] 10.222.0.45 () {56 vars in 1124 bytes} [Wed Jun 15 14:29:55 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:56.061285 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f4a0f0-d705-4f5d-b136-d3d8e0e8f091 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:56.214031 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f4a0f0-d705-4f5d-b136-d3d8e0e8f091 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:56.247671 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:56.251940 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c7f91-69ef-477a-8d58-996fd92669a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:56.252724 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c7f91-69ef-477a-8d58-996fd92669a2 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:56.253235 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c7f91-69ef-477a-8d58-996fd92669a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:56.253571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c7f91-69ef-477a-8d58-996fd92669a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:56.254451 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3c7f91-69ef-477a-8d58-996fd92669a2 None admin] RBAC: Adding query filter params (interface=public, service_id=eab0573d2e6249ab810125aaa17bed01, region_id=RegionOne) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:56.254842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3c7f91-69ef-477a-8d58-996fd92669a2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:56.256312 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3c7f91-69ef-477a-8d58-996fd92669a2 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:56.262845 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 385/779] 10.222.0.45 () {56 vars in 1251 bytes} [Wed Jun 15 14:29:56 2022] GET /identity/v3/endpoints?service_id=eab0573d2e6249ab810125aaa17bed01&interface=public®ion_id=RegionOne => generated 194 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:56.268303 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6884d085-60f8-42d1-9c80-e4b018f81ee0 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:56.277864 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1fc22-59c6-4dd9-a5c3-b76ab6959f2e None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:56.278244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1fc22-59c6-4dd9-a5c3-b76ab6959f2e None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:56.278598 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1fc22-59c6-4dd9-a5c3-b76ab6959f2e None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:56.280324 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 386/780] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:29:56.287485 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7bcdd-596c-4814-b561-e69551561780 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:56.287855 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7bcdd-596c-4814-b561-e69551561780 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:56.288230 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7bcdd-596c-4814-b561-e69551561780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:56.305079 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-fed7bcdd-596c-4814-b561-e69551561780 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:56.306388 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-fed7bcdd-596c-4814-b561-e69551561780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:56.376675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 387/781] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:29:56.415533 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed7bcdd-596c-4814-b561-e69551561780 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:56.447341 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6884d085-60f8-42d1-9c80-e4b018f81ee0 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:56.485467 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f22bbe-3e89-4d00-b77e-e567a3fefd01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:56.486272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f22bbe-3e89-4d00-b77e-e567a3fefd01 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:56.486572 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f22bbe-3e89-4d00-b77e-e567a3fefd01 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:56.486858 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f22bbe-3e89-4d00-b77e-e567a3fefd01 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:56.487911 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f22bbe-3e89-4d00-b77e-e567a3fefd01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:56.489458 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f22bbe-3e89-4d00-b77e-e567a3fefd01 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:56.495384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 395/782] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Jun 15 14:29:56 2022] GET /identity/v3/services => generated 772 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:56.618135 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed7bcdd-596c-4814-b561-e69551561780 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:56.661809 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:56.672049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a2467-77f2-4d7c-aac0-09f8c7c4e06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:56.673093 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a2467-77f2-4d7c-aac0-09f8c7c4e06b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:56.673353 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a2467-77f2-4d7c-aac0-09f8c7c4e06b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:56.673529 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a2467-77f2-4d7c-aac0-09f8c7c4e06b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:56.692991 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187a2467-77f2-4d7c-aac0-09f8c7c4e06b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:56.701527 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-187a2467-77f2-4d7c-aac0-09f8c7c4e06b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:29:56.701527 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:56.701527 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:56.701527 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:56.701527 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:56.701527 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:56.701527 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:56.701527 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:56.701527 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:56.701527 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:56.701527 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:56.701527 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:56.701527 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:56.701527 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:56.701527 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:56.701527 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:56.703269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:56.703269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:56.703269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:56.703269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:56.703269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:56.703269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:56.703269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:56.703269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:56.703269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:56.703269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:56.703269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:56.703269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:56.703269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:56.703269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:56.703269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:56.703269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:56.704798 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:56.704798 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:56.704798 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:56.704798 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:56.704798 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:56.704798 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:56.704798 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:56.704798 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:56.704798 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:56.704798 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:56.704798 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:29:56.704798 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:56.704798 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 388/783] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:29:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:29:56.720789 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f22bbe-3e89-4d00-b77e-e567a3fefd01 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:56.800499 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:56.899994 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f22bbe-3e89-4d00-b77e-e567a3fefd01 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:56.958140 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7785bda7-fdc6-43d3-9459-fcb241c81f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:56.959224 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785bda7-fdc6-43d3-9459-fcb241c81f21 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:56.959766 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785bda7-fdc6-43d3-9459-fcb241c81f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:56.960241 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785bda7-fdc6-43d3-9459-fcb241c81f21 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:56.961611 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7785bda7-fdc6-43d3-9459-fcb241c81f21 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:56.962157 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7785bda7-fdc6-43d3-9459-fcb241c81f21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:56.964691 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7785bda7-fdc6-43d3-9459-fcb241c81f21 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:56.979357 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 396/784] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:56.985287 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a2467-77f2-4d7c-aac0-09f8c7c4e06b None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:57.090962 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a2467-77f2-4d7c-aac0-09f8c7c4e06b None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:57.126646 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4416414-7b52-41b6-a8f8-e8b38e682034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:57.127488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4416414-7b52-41b6-a8f8-e8b38e682034 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:57.127899 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4416414-7b52-41b6-a8f8-e8b38e682034 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:57.128175 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4416414-7b52-41b6-a8f8-e8b38e682034 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:57.139253 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4416414-7b52-41b6-a8f8-e8b38e682034 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:57.140727 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4416414-7b52-41b6-a8f8-e8b38e682034 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:57.158284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-f4416414-7b52-41b6-a8f8-e8b38e682034 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:57.158284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:57.158284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:57.158284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:57.158284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:57.158284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:57.158284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:57.158284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:57.158284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:57.158284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:57.158284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:57.158284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:57.158284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:57.158284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:57.158284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:57.159228 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:57.159228 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:57.159228 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:57.159228 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:57.159228 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:57.159228 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:57.159228 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:57.159228 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:57.159228 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:57.159228 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:57.159228 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:57.159228 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:57.159228 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:57.159228 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:57.159228 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:57.159228 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:57.159228 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:57.160115 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:57.160115 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:57.160115 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:57.160115 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:57.160115 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 389/785] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:29:57.168638 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7785bda7-fdc6-43d3-9459-fcb241c81f21 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:57.301565 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7785bda7-fdc6-43d3-9459-fcb241c81f21 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:57.304094 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:57.350177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4aa41c-2de2-433d-b42b-c06c07697b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:57.350849 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4aa41c-2de2-433d-b42b-c06c07697b53 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:57.351161 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4aa41c-2de2-433d-b42b-c06c07697b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:57.351445 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4aa41c-2de2-433d-b42b-c06c07697b53 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:57.352173 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4aa41c-2de2-433d-b42b-c06c07697b53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:57.352484 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4aa41c-2de2-433d-b42b-c06c07697b53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:57.353861 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4aa41c-2de2-433d-b42b-c06c07697b53 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:57.366655 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 397/786] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:57.372609 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4416414-7b52-41b6-a8f8-e8b38e682034 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:57.434568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:57.506911 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4416414-7b52-41b6-a8f8-e8b38e682034 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:57.546396 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d45f5-887e-4655-bc93-8d8a884e239e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:57.547256 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d45f5-887e-4655-bc93-8d8a884e239e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:57.547547 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d45f5-887e-4655-bc93-8d8a884e239e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:57.547823 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d45f5-887e-4655-bc93-8d8a884e239e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:57.555984 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280d45f5-887e-4655-bc93-8d8a884e239e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:57.557446 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280d45f5-887e-4655-bc93-8d8a884e239e None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:57.577330 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-280d45f5-887e-4655-bc93-8d8a884e239e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:29:57.577330 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:57.577330 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:57.577330 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:57.577330 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:57.577330 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:57.577330 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:57.577330 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:57.577330 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:57.577330 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:57.577330 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:57.577330 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:57.577330 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:29:57.577330 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:29:57.577330 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:29:57.577330 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:29:57.578307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:57.578307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:57.578307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:29:57.578307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:57.578307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:29:57.578307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:57.578307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:57.578307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:57.578307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:57.578307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:57.578307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:57.578307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:57.578307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:57.578307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:57.578307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:57.578307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:57.578307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:57.578307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:57.579254 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:57.579254 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:57.579254 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:29:57.579254 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:29:57.579254 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:29:57.579254 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:29:57.579254 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:29:57.579254 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:29:57.579254 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:29:57.579254 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:57.579254 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 390/787] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:29:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:57.585901 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4aa41c-2de2-433d-b42b-c06c07697b53 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:57.746554 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4aa41c-2de2-433d-b42b-c06c07697b53 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:57.751423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3088c43-71af-4a89-a088-219eddb7e910 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:57.751711 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3088c43-71af-4a89-a088-219eddb7e910 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:57.751989 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3088c43-71af-4a89-a088-219eddb7e910 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:57.753370 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 391/788] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:29:57.759960 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df4fd4-faf4-4ed1-a55d-5250ae97b5f6 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:57.760247 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df4fd4-faf4-4ed1-a55d-5250ae97b5f6 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:57.760528 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df4fd4-faf4-4ed1-a55d-5250ae97b5f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:57.794236 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-12df4fd4-faf4-4ed1-a55d-5250ae97b5f6 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:57.796700 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-12df4fd4-faf4-4ed1-a55d-5250ae97b5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:57.801632 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e6cea-5e05-4b9c-b00d-39453621ea68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:57.802511 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e6cea-5e05-4b9c-b00d-39453621ea68 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:57.802965 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e6cea-5e05-4b9c-b00d-39453621ea68 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:57.803457 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e6cea-5e05-4b9c-b00d-39453621ea68 None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:57.804461 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7e6cea-5e05-4b9c-b00d-39453621ea68 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:57.804937 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7e6cea-5e05-4b9c-b00d-39453621ea68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:57.806540 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7e6cea-5e05-4b9c-b00d-39453621ea68 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:57.843513 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 398/789] 10.222.0.45 () {56 vars in 1135 bytes} [Wed Jun 15 14:29:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:57.849762 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e6cea-5e05-4b9c-b00d-39453621ea68 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:57.893722 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:57.935620 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 392/790] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:29:57.996388 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df4fd4-faf4-4ed1-a55d-5250ae97b5f6 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:58.018759 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e6cea-5e05-4b9c-b00d-39453621ea68 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:58.031460 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:58.078502 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f8791-4742-4900-b766-2f6300f1aa04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:58.079754 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f8791-4742-4900-b766-2f6300f1aa04 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:58.081210 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f8791-4742-4900-b766-2f6300f1aa04 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:58.081830 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f8791-4742-4900-b766-2f6300f1aa04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:58.102879 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f8791-4742-4900-b766-2f6300f1aa04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:58.104661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f8791-4742-4900-b766-2f6300f1aa04 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:58.133693 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-895f8791-4742-4900-b766-2f6300f1aa04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:58.133693 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:58.133693 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:58.133693 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:58.133693 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:58.133693 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:58.133693 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:58.133693 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:58.133693 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:58.133693 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:58.133693 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:58.133693 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:58.133693 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:58.133693 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:58.133693 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:58.134615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:58.134615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:58.134615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:58.134615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:58.134615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:58.134615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:58.134615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:58.134615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:58.134615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:58.134615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:58.134615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:58.134615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:58.134615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:58.134615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:58.134615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:58.134615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:58.134615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:58.135528 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:58.135528 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:58.135528 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:58.135528 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:58.135528 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 399/791] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:58.141480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f8791-4742-4900-b766-2f6300f1aa04 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:58.171837 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df4fd4-faf4-4ed1-a55d-5250ae97b5f6 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:58.240877 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503d0b6-e43c-4acc-bb9b-6d8e72b09b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:58.241682 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503d0b6-e43c-4acc-bb9b-6d8e72b09b33 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:58.241964 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503d0b6-e43c-4acc-bb9b-6d8e72b09b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:58.242241 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503d0b6-e43c-4acc-bb9b-6d8e72b09b33 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:58.261232 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9503d0b6-e43c-4acc-bb9b-6d8e72b09b33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:58.271810 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-9503d0b6-e43c-4acc-bb9b-6d8e72b09b33 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:58.271810 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:58.271810 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:58.271810 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:58.271810 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:58.271810 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:58.271810 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:58.271810 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:58.271810 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:58.271810 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:58.271810 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:58.271810 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:58.271810 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:58.271810 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:58.271810 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:58.271810 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:58.272800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:58.272800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:58.272800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:58.272800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:58.272800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:58.272800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:58.272800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:58.272800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:58.272800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:58.272800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:58.272800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:58.272800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:58.272800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:58.272800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:58.272800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:58.272800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:58.273598 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:58.273598 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:58.273598 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:58.273598 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:58.273598 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:58.273598 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:58.273598 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:58.273598 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:58.273598 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:58.273598 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:58.273598 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:58.273598 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:58.273598 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 393/792] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Jun 15 14:29:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:29:58.281429 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503d0b6-e43c-4acc-bb9b-6d8e72b09b33 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:58.284972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:58.341051 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f8791-4742-4900-b766-2f6300f1aa04 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:58.394290 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09595a6c-b41f-4eeb-8fc6-c2a0e088e8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:58.395245 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09595a6c-b41f-4eeb-8fc6-c2a0e088e8da None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:58.395536 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09595a6c-b41f-4eeb-8fc6-c2a0e088e8da None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:58.395818 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09595a6c-b41f-4eeb-8fc6-c2a0e088e8da None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:58.396538 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09595a6c-b41f-4eeb-8fc6-c2a0e088e8da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:58.396852 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09595a6c-b41f-4eeb-8fc6-c2a0e088e8da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:58.398339 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09595a6c-b41f-4eeb-8fc6-c2a0e088e8da None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:58.410901 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 400/793] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:58.412227 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee67367-a6a8-4eef-aa26-d9720e725672 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:58.412521 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee67367-a6a8-4eef-aa26-d9720e725672 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:58.412834 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee67367-a6a8-4eef-aa26-d9720e725672 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:58.418253 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 401/794] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:58.419462 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d96cd-39fc-41f5-8fee-4ed3dea0f696 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:58.419755 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d96cd-39fc-41f5-8fee-4ed3dea0f696 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:58.420037 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d96cd-39fc-41f5-8fee-4ed3dea0f696 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:58.421286 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 402/795] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:58.421988 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d96cd-39fc-41f5-8fee-4ed3dea0f696 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:58.428992 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:58.444891 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503d0b6-e43c-4acc-bb9b-6d8e72b09b33 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:58.488930 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bdf376-d03e-46b8-9a46-902ca8a60254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:58.489597 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bdf376-d03e-46b8-9a46-902ca8a60254 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:58.489877 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bdf376-d03e-46b8-9a46-902ca8a60254 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:58.490156 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bdf376-d03e-46b8-9a46-902ca8a60254 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:58.490893 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bdf376-d03e-46b8-9a46-902ca8a60254 None admin] RBAC: Adding query filter params (name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:58.491229 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bdf376-d03e-46b8-9a46-902ca8a60254 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:58.492697 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bdf376-d03e-46b8-9a46-902ca8a60254 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:58.500708 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 394/796] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Jun 15 14:29:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:58.504281 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0817da-81ca-482f-b0d1-c545f2e18572 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:58.504569 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0817da-81ca-482f-b0d1-c545f2e18572 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:58.504848 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0817da-81ca-482f-b0d1-c545f2e18572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:58.525041 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-bc0817da-81ca-482f-b0d1-c545f2e18572 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:58.526365 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-bc0817da-81ca-482f-b0d1-c545f2e18572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:58.533840 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d96cd-39fc-41f5-8fee-4ed3dea0f696 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:58.570555 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefdcda6-4653-44c3-b54c-e82d944efafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:58.571373 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefdcda6-4653-44c3-b54c-e82d944efafc None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:58.571652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefdcda6-4653-44c3-b54c-e82d944efafc None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:58.571921 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefdcda6-4653-44c3-b54c-e82d944efafc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:58.574374 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:58.580306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefdcda6-4653-44c3-b54c-e82d944efafc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:58.582060 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefdcda6-4653-44c3-b54c-e82d944efafc None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:58.597254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-fefdcda6-4653-44c3-b54c-e82d944efafc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:58.597254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:58.597254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:58.597254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:58.597254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:58.597254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:58.597254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:58.597254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:58.597254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:58.597254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:58.597254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:58.597254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:58.597254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:29:58.597254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:29:58.597254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:29:58.598735 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:29:58.598735 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:58.598735 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:58.598735 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:58.598735 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:58.598735 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:58.598735 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:58.598735 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:58.598735 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:58.598735 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:58.598735 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:58.598735 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:58.598735 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:58.598735 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:58.598735 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:58.598735 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:58.598735 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:58.599738 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:29:58.599738 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:29:58.599738 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:29:58.599738 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:29:58.599738 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:29:58.599738 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:29:58.599738 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:58.599738 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:58.599738 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 403/797] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:58.600274 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc32b8-5b15-4e25-8046-70910576ba29 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:58.600523 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc32b8-5b15-4e25-8046-70910576ba29 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:58.600797 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc32b8-5b15-4e25-8046-70910576ba29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:58.609408 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 395/798] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:29:58.610533 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0817da-81ca-482f-b0d1-c545f2e18572 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:58.617512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-d4bc32b8-5b15-4e25-8046-70910576ba29 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:58.618792 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d4bc32b8-5b15-4e25-8046-70910576ba29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:58.690601 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 404/799] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:29:58.692329 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc32b8-5b15-4e25-8046-70910576ba29 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:58.706171 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:58.734652 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0817da-81ca-482f-b0d1-c545f2e18572 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:58.772332 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b31aa-aff3-46f9-8500-76105f9cd873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:58.773454 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b31aa-aff3-46f9-8500-76105f9cd873 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:58.773757 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b31aa-aff3-46f9-8500-76105f9cd873 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:58.774016 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b31aa-aff3-46f9-8500-76105f9cd873 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:58.785150 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568b31aa-aff3-46f9-8500-76105f9cd873 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:58.787942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568b31aa-aff3-46f9-8500-76105f9cd873 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:58.805326 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc32b8-5b15-4e25-8046-70910576ba29 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:58.808753 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-568b31aa-aff3-46f9-8500-76105f9cd873 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:58.808753 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:58.808753 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:58.808753 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:58.808753 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:58.808753 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:58.808753 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:58.808753 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:58.808753 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:58.808753 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:58.808753 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:58.808753 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:58.808753 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:58.808753 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:58.808753 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:58.809703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:58.809703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:58.809703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:58.809703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:58.809703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:58.809703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:58.809703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:58.809703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:58.809703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:58.809703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:58.809703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:58.809703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:58.809703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:58.809703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:58.809703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:58.809703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:58.809703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:58.810568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:58.810568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:58.810568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:58.810568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:58.810568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 396/800] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:58.810568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b31aa-aff3-46f9-8500-76105f9cd873 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:58.835727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:58.847716 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7a88b-c388-45c2-a0bb-950e935d85f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:58.848300 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7a88b-c388-45c2-a0bb-950e935d85f7 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:58.848586 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7a88b-c388-45c2-a0bb-950e935d85f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:58.848865 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7a88b-c388-45c2-a0bb-950e935d85f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:58.849654 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca7a88b-c388-45c2-a0bb-950e935d85f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:58.849965 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca7a88b-c388-45c2-a0bb-950e935d85f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:58.851443 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca7a88b-c388-45c2-a0bb-950e935d85f7 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:58.863350 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 405/801] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:29:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:58.867955 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7a88b-c388-45c2-a0bb-950e935d85f7 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:58.918488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b31aa-aff3-46f9-8500-76105f9cd873 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:58.949674 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:58.956645 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ebf4ca-5ac1-4a4a-ac38-bb978c56f7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:58.957548 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ebf4ca-5ac1-4a4a-ac38-bb978c56f7ec None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:58.957942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ebf4ca-5ac1-4a4a-ac38-bb978c56f7ec None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:58.958222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ebf4ca-5ac1-4a4a-ac38-bb978c56f7ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:58.969286 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ebf4ca-5ac1-4a4a-ac38-bb978c56f7ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:58.972417 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7a88b-c388-45c2-a0bb-950e935d85f7 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:58.977282 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-84ebf4ca-5ac1-4a4a-ac38-bb978c56f7ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:29:58.977282 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:58.977282 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:58.977282 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:58.977282 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:58.977282 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:58.977282 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:58.977282 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:58.977282 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:58.977282 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:58.977282 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:58.977282 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:58.977282 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:58.977282 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:58.977282 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:58.977282 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:58.978230 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:58.978230 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:58.978230 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:58.978230 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:58.978230 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:58.978230 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:58.978230 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:58.978230 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:58.978230 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:58.978230 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:58.978230 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:58.978230 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:58.978230 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:58.978230 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:58.978230 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:58.978230 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:58.983939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:58.983939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:58.983939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:58.983939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:58.983939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:58.983939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:58.983939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:58.983939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:58.983939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:58.983939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:58.983939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:29:58.983939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:58.983939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 397/802] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:29:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:58.983939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ebf4ca-5ac1-4a4a-ac38-bb978c56f7ec None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:59.015737 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac02a4c-4fef-4d16-997f-526d9ac76f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:59.016536 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac02a4c-4fef-4d16-997f-526d9ac76f2f None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:59.016831 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac02a4c-4fef-4d16-997f-526d9ac76f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:59.017092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac02a4c-4fef-4d16-997f-526d9ac76f2f None admin] PATH_INFO: `/v3/services/network` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:59.017782 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac02a4c-4fef-4d16-997f-526d9ac76f2f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:59.019415 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac02a4c-4fef-4d16-997f-526d9ac76f2f None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:59.032177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-2ac02a4c-4fef-4d16-997f-526d9ac76f2f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:29:59.032177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:59.032177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:29:59.032177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:29:59.032177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:29:59.032177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:29:59.032177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:29:59.032177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:29:59.032177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:29:59.032177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:59.032177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:29:59.032177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:59.032177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:59.032177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:59.032177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:59.032177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:59.032177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:59.033179 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:59.033179 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:59.033179 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:59.033179 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:59.033179 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:59.033179 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:59.033179 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:29:59.033179 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:29:59.033179 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:29:59.033179 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:29:59.033179 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:59.033179 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:59.033179 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:59.033179 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:59.033179 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:59.033179 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:59.034037 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:59.034037 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:59.034037 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:59.034037 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:59.034037 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:59.034037 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:59.034037 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:59.034037 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:59.034037 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:59.034037 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:59.034037 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:29:59.034037 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:29:59.034037 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:29:59.034037 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:59.034037 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 406/803] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:58 2022] GET /identity/v3/services/network => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:59.034811 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac02a4c-4fef-4d16-997f-526d9ac76f2f None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:59.087799 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:59.096262 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ebf4ca-5ac1-4a4a-ac38-bb978c56f7ec None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:59.135205 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8184c6e-a3d6-4baf-897a-b173051c26ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:59.135875 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8184c6e-a3d6-4baf-897a-b173051c26ec None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:59.136163 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8184c6e-a3d6-4baf-897a-b173051c26ec None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:59.137003 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8184c6e-a3d6-4baf-897a-b173051c26ec None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:59.137730 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8184c6e-a3d6-4baf-897a-b173051c26ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:59.138048 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8184c6e-a3d6-4baf-897a-b173051c26ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:59.139456 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8184c6e-a3d6-4baf-897a-b173051c26ec None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:59.146464 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac02a4c-4fef-4d16-997f-526d9ac76f2f None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:59.151559 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 398/804] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:59.152789 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8184c6e-a3d6-4baf-897a-b173051c26ec None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:59.189922 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be07d38-9c30-46c9-bab5-99e7c110d5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:59.190809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be07d38-9c30-46c9-bab5-99e7c110d5e8 None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:59.191136 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be07d38-9c30-46c9-bab5-99e7c110d5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:59.191416 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be07d38-9c30-46c9-bab5-99e7c110d5e8 None admin] PATH_INFO: `/v3/projects/d488c79cb1704e9496fc196d8aa6c104/users/641a81b3140f47119b5c2323a97a6681/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:59.193395 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be07d38-9c30-46c9-bab5-99e7c110d5e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=d488c79cb1704e9496fc196d8aa6c104, user_id=641a81b3140f47119b5c2323a97a6681, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:59.194880 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be07d38-9c30-46c9-bab5-99e7c110d5e8 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:59.208095 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 407/805] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Jun 15 14:29:59 2022] PUT /identity/v3/projects/d488c79cb1704e9496fc196d8aa6c104/users/641a81b3140f47119b5c2323a97a6681/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:29:59.208883 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be07d38-9c30-46c9-bab5-99e7c110d5e8 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:59.210138 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:59.276518 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8184c6e-a3d6-4baf-897a-b173051c26ec None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:59.316735 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963767d-7b8d-4db9-a935-fc8bf609daa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:59.317602 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963767d-7b8d-4db9-a935-fc8bf609daa8 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:59.317978 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963767d-7b8d-4db9-a935-fc8bf609daa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:59.318307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963767d-7b8d-4db9-a935-fc8bf609daa8 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:59.319115 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b963767d-7b8d-4db9-a935-fc8bf609daa8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:59.319432 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b963767d-7b8d-4db9-a935-fc8bf609daa8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:59.320842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b963767d-7b8d-4db9-a935-fc8bf609daa8 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:59.324079 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be07d38-9c30-46c9-bab5-99e7c110d5e8 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:59.335289 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 399/806] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:59.335849 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:59.336833 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68c939-4792-41f4-b88a-f8f0335dcf18 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:59.337034 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68c939-4792-41f4-b88a-f8f0335dcf18 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:59.337427 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68c939-4792-41f4-b88a-f8f0335dcf18 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:59.339152 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 400/807] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:59.339583 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68c939-4792-41f4-b88a-f8f0335dcf18 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:59.373241 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca036fa3-de6d-4278-8868-37ddc834504a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:59.374247 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca036fa3-de6d-4278-8868-37ddc834504a None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:59.374759 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca036fa3-de6d-4278-8868-37ddc834504a None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:59.375272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca036fa3-de6d-4278-8868-37ddc834504a None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:59.376212 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca036fa3-de6d-4278-8868-37ddc834504a None admin] RBAC: Adding query filter params (name=network) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:59.376678 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca036fa3-de6d-4278-8868-37ddc834504a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:59.378202 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca036fa3-de6d-4278-8868-37ddc834504a None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:59.385113 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 408/808] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Jun 15 14:29:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:59.386116 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca036fa3-de6d-4278-8868-37ddc834504a None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:59.465062 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68c939-4792-41f4-b88a-f8f0335dcf18 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:59.483902 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:59.490829 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca036fa3-de6d-4278-8868-37ddc834504a None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:59.505193 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5474c1b-d264-4df4-8202-833082eb2cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:59.506003 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5474c1b-d264-4df4-8202-833082eb2cff None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:59.506341 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5474c1b-d264-4df4-8202-833082eb2cff None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:59.506876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5474c1b-d264-4df4-8202-833082eb2cff None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:59.514666 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5474c1b-d264-4df4-8202-833082eb2cff None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:59.516169 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5474c1b-d264-4df4-8202-833082eb2cff None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:59.527952 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58eeee6-f3b2-4883-a80a-cc0bd9302170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:59.528748 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58eeee6-f3b2-4883-a80a-cc0bd9302170 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:59.529047 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58eeee6-f3b2-4883-a80a-cc0bd9302170 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:59.529535 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58eeee6-f3b2-4883-a80a-cc0bd9302170 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:59.532104 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-f5474c1b-d264-4df4-8202-833082eb2cff None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:29:59.532104 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:59.532104 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:59.532104 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:59.532104 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:59.532104 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:59.532104 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:59.532104 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:59.532104 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:59.532104 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:59.532104 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:59.532104 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:59.532104 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:29:59.532104 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:29:59.532104 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:29:59.532104 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:29:59.533013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:59.533013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:59.533013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:29:59.533013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:59.533013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:29:59.533013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:59.533013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:59.533013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:59.533013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:59.533013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:59.533013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:59.533013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:59.533013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:59.533013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:59.533013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:59.533013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:59.533013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:59.533013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:59.533974 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:59.533974 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:59.533974 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:29:59.533974 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:29:59.533974 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:29:59.533974 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:29:59.533974 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:29:59.533974 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:29:59.533974 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:29:59.533974 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:29:59.533974 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 401/809] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Jun 15 14:29:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:59.535255 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494a203-b04d-4766-99b0-ddb8c82f9c4f None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:59.535511 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494a203-b04d-4766-99b0-ddb8c82f9c4f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:59.536011 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494a203-b04d-4766-99b0-ddb8c82f9c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:59.549346 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58eeee6-f3b2-4883-a80a-cc0bd9302170 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:59.550813 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58eeee6-f3b2-4883-a80a-cc0bd9302170 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:59.554236 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-4494a203-b04d-4766-99b0-ddb8c82f9c4f None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:59.555480 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-4494a203-b04d-4766-99b0-ddb8c82f9c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:59.568521 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-d58eeee6-f3b2-4883-a80a-cc0bd9302170 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:59.568521 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:59.568521 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:59.568521 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:59.568521 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:59.568521 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:59.568521 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:59.568521 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:59.568521 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:59.568521 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:59.568521 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:59.568521 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:59.568521 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:59.568521 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:59.568521 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:59.569557 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:59.569557 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:59.569557 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:59.569557 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:59.569557 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:59.569557 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:59.569557 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:59.569557 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:59.569557 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:59.569557 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:59.569557 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:59.569557 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:59.569557 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:59.569557 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:59.569557 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:59.569557 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:59.569557 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:59.570443 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:59.570443 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:59.570443 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:59.570443 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:29:59.570443 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 409/810] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:59.571197 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58eeee6-f3b2-4883-a80a-cc0bd9302170 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:59.597336 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:59.643378 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 402/811] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:29:59.645440 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494a203-b04d-4766-99b0-ddb8c82f9c4f None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:59.681450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58eeee6-f3b2-4883-a80a-cc0bd9302170 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:59.717897 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3276ed-3933-4c9b-8688-7dd349618371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:59.718567 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3276ed-3933-4c9b-8688-7dd349618371 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:59.718852 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3276ed-3933-4c9b-8688-7dd349618371 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:59.719153 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3276ed-3933-4c9b-8688-7dd349618371 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:59.719864 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3276ed-3933-4c9b-8688-7dd349618371 None admin] RBAC: Adding query filter params (type=network) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:59.720173 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3276ed-3933-4c9b-8688-7dd349618371 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:59.721548 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3276ed-3933-4c9b-8688-7dd349618371 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:59.731880 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 410/812] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Jun 15 14:29:59 2022] GET /identity/v3/services?type=network => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:59.732627 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3276ed-3933-4c9b-8688-7dd349618371 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:59.768566 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494a203-b04d-4766-99b0-ddb8c82f9c4f None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:59.771207 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:59.806970 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239f9b63-7ffa-425d-b4d9-a8f72e4fe193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:59.807892 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f9b63-7ffa-425d-b4d9-a8f72e4fe193 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:59.808873 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f9b63-7ffa-425d-b4d9-a8f72e4fe193 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:59.808873 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f9b63-7ffa-425d-b4d9-a8f72e4fe193 None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:59.809570 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239f9b63-7ffa-425d-b4d9-a8f72e4fe193 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:59.810140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239f9b63-7ffa-425d-b4d9-a8f72e4fe193 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:59.811993 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239f9b63-7ffa-425d-b4d9-a8f72e4fe193 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:59.827048 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:59.836940 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 403/813] 10.222.0.45 () {56 vars in 1149 bytes} [Wed Jun 15 14:29:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:59.837933 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239f9b63-7ffa-425d-b4d9-a8f72e4fe193 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:59.840564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3276ed-3933-4c9b-8688-7dd349618371 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:59.877575 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0139b76-b3db-438a-8c27-9bdc04c88559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:59.878172 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0139b76-b3db-438a-8c27-9bdc04c88559 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:59.878458 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0139b76-b3db-438a-8c27-9bdc04c88559 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:59.878741 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0139b76-b3db-438a-8c27-9bdc04c88559 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:59.879507 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0139b76-b3db-438a-8c27-9bdc04c88559 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:59.879817 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0139b76-b3db-438a-8c27-9bdc04c88559 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:59.881208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0139b76-b3db-438a-8c27-9bdc04c88559 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:59.898833 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 411/814] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:59.898833 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0139b76-b3db-438a-8c27-9bdc04c88559 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:59.947313 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239f9b63-7ffa-425d-b4d9-a8f72e4fe193 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:59.983524 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd2471-ab22-4520-8583-adb777235206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:59.984337 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd2471-ab22-4520-8583-adb777235206 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:59.984626 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd2471-ab22-4520-8583-adb777235206 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:59.984910 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd2471-ab22-4520-8583-adb777235206 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:59.985562 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd2471-ab22-4520-8583-adb777235206 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:59.986941 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd2471-ab22-4520-8583-adb777235206 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:59.999979 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0139b76-b3db-438a-8c27-9bdc04c88559 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:00.001050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-6abd2471-ab22-4520-8583-adb777235206 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:30:00.001050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:00.001050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:00.001050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:00.001050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:00.001050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:00.001050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:00.001050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:00.001050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:30:00.001050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:00.001050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:00.001050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:00.001050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:00.001050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:00.001050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:00.001050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:00.001934 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:00.001934 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:00.001934 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:00.001934 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:00.001934 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:00.001934 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:00.001934 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:00.001934 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:00.001934 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:00.001934 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:00.001934 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:00.001934 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:00.001934 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:00.001934 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:00.001934 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:00.001934 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:00.002776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:00.002776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:00.002776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:00.002776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:00.002776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:00.002776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:00.002776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:00.002776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:00.002776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:00.002776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:00.002776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:00.002776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:00.002776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:00.002776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:30:00.002776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:00.002776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 404/815] 10.222.0.45 () {56 vars in 1129 bytes} [Wed Jun 15 14:29:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:00.003626 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd2471-ab22-4520-8583-adb777235206 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:00.037100 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:00.042928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d152c9c-692d-48b6-a43c-afefdb44763b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:00.043739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d152c9c-692d-48b6-a43c-afefdb44763b None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:00.044123 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d152c9c-692d-48b6-a43c-afefdb44763b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:00.044497 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d152c9c-692d-48b6-a43c-afefdb44763b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:00.045327 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d152c9c-692d-48b6-a43c-afefdb44763b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:00.046806 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d152c9c-692d-48b6-a43c-afefdb44763b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:00.069200 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 412/816] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Jun 15 14:29:59 2022] POST /identity/v3/endpoints => generated 327 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:00.070274 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d152c9c-692d-48b6-a43c-afefdb44763b None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:00.080709 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:00.134668 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd2471-ab22-4520-8583-adb777235206 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:00.182785 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daca9b8-01cc-43a4-9f47-a341d6e6227e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:00.183774 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daca9b8-01cc-43a4-9f47-a341d6e6227e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:00.184131 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daca9b8-01cc-43a4-9f47-a341d6e6227e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:00.184480 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daca9b8-01cc-43a4-9f47-a341d6e6227e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:00.197305 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daca9b8-01cc-43a4-9f47-a341d6e6227e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:00.199619 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daca9b8-01cc-43a4-9f47-a341d6e6227e None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:00.208687 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d152c9c-692d-48b6-a43c-afefdb44763b None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:00.219917 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-9daca9b8-01cc-43a4-9f47-a341d6e6227e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:00.219917 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:00.219917 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:00.219917 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:00.219917 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:00.219917 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:00.219917 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:00.219917 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:00.219917 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:00.219917 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:00.219917 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:00.219917 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:00.219917 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:00.219917 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:00.219917 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:00.221047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:00.221047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:00.221047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:00.221047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:00.221047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:00.221047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:00.221047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:00.221047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:00.221047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:00.221047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:00.221047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:00.221047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:00.221047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:00.221047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:00.221047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:00.221047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:00.221047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:00.222111 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:00.222111 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:00.222111 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:00.222111 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:00.227802 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 405/817] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:00.227802 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daca9b8-01cc-43a4-9f47-a341d6e6227e None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:00.263630 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cbab7e-8b6e-4a21-9302-d7d4554f862b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:00.264593 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cbab7e-8b6e-4a21-9302-d7d4554f862b None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:00.265777 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cbab7e-8b6e-4a21-9302-d7d4554f862b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:00.266215 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cbab7e-8b6e-4a21-9302-d7d4554f862b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:00.275586 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cbab7e-8b6e-4a21-9302-d7d4554f862b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:00.278419 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cbab7e-8b6e-4a21-9302-d7d4554f862b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:00.294928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-54cbab7e-8b6e-4a21-9302-d7d4554f862b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:30:00.294928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:00.294928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:00.294928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:00.294928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:00.294928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:00.294928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:00.294928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:00.294928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:00.294928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:00.294928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:00.294928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:00.294928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:30:00.294928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:30:00.294928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:30:00.294928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:30:00.296140 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:00.296140 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:00.296140 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:00.296140 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:00.296140 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:00.296140 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:00.296140 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:00.296140 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:00.296140 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:00.296140 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:00.296140 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:00.296140 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:00.296140 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:00.296140 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:00.296140 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:00.296140 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:00.296140 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:00.296140 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:00.297259 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:00.297259 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:00.297259 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:30:00.297259 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:30:00.297259 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:30:00.297259 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:30:00.297259 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:30:00.297259 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:30:00.297259 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:30:00.297259 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:00.302530 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 413/818] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Jun 15 14:30:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:00.302530 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cbab7e-8b6e-4a21-9302-d7d4554f862b None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:00.327580 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:00.357301 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daca9b8-01cc-43a4-9f47-a341d6e6227e None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:00.379646 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:00.394551 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0958c9d2-9c16-4706-ae87-5e12bc88ef25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:00.395452 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958c9d2-9c16-4706-ae87-5e12bc88ef25 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:00.395735 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958c9d2-9c16-4706-ae87-5e12bc88ef25 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:00.396007 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958c9d2-9c16-4706-ae87-5e12bc88ef25 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:00.396710 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0958c9d2-9c16-4706-ae87-5e12bc88ef25 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:00.397010 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0958c9d2-9c16-4706-ae87-5e12bc88ef25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:00.398367 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0958c9d2-9c16-4706-ae87-5e12bc88ef25 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:00.407596 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 406/819] 10.222.0.45 () {56 vars in 1124 bytes} [Wed Jun 15 14:30:00 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:00.408324 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0958c9d2-9c16-4706-ae87-5e12bc88ef25 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:00.422637 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cbab7e-8b6e-4a21-9302-d7d4554f862b None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:00.460405 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388f9a2-f6cd-424d-9c36-d5e297855888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:00.461255 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388f9a2-f6cd-424d-9c36-d5e297855888 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:00.461642 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388f9a2-f6cd-424d-9c36-d5e297855888 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:00.462019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388f9a2-f6cd-424d-9c36-d5e297855888 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:00.462851 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6388f9a2-f6cd-424d-9c36-d5e297855888 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:00.463288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6388f9a2-f6cd-424d-9c36-d5e297855888 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:00.464795 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6388f9a2-f6cd-424d-9c36-d5e297855888 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:00.481791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 414/820] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:00.481791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388f9a2-f6cd-424d-9c36-d5e297855888 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:00.521726 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0958c9d2-9c16-4706-ae87-5e12bc88ef25 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:00.558053 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6e303c-49dc-4eb4-9225-0a11c49ea426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:00.559284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e303c-49dc-4eb4-9225-0a11c49ea426 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:00.559670 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e303c-49dc-4eb4-9225-0a11c49ea426 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:00.559950 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e303c-49dc-4eb4-9225-0a11c49ea426 None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:00.560906 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6e303c-49dc-4eb4-9225-0a11c49ea426 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:00.561221 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6e303c-49dc-4eb4-9225-0a11c49ea426 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:00.562675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6e303c-49dc-4eb4-9225-0a11c49ea426 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:00.580839 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388f9a2-f6cd-424d-9c36-d5e297855888 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:00.606381 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:00.618284 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52b1b9-dbfb-49ac-9bb2-a39e655848ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:00.618972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52b1b9-dbfb-49ac-9bb2-a39e655848ce None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:00.619319 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52b1b9-dbfb-49ac-9bb2-a39e655848ce None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:00.619597 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52b1b9-dbfb-49ac-9bb2-a39e655848ce None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:00.620327 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d52b1b9-dbfb-49ac-9bb2-a39e655848ce None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:00.620638 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d52b1b9-dbfb-49ac-9bb2-a39e655848ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:00.622690 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d52b1b9-dbfb-49ac-9bb2-a39e655848ce None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:00.634605 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 415/821] 10.222.0.45 () {56 vars in 1124 bytes} [Wed Jun 15 14:30:00 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:00.634605 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52b1b9-dbfb-49ac-9bb2-a39e655848ce None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:00.691367 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:00.707124 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 407/822] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Jun 15 14:30:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:00.707124 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6e303c-49dc-4eb4-9225-0a11c49ea426 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:00.748476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52b1b9-dbfb-49ac-9bb2-a39e655848ce None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:00.784273 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76813ee8-3124-42a9-bc0b-7179d4f408bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:00.785067 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76813ee8-3124-42a9-bc0b-7179d4f408bc None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:00.785355 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76813ee8-3124-42a9-bc0b-7179d4f408bc None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:00.785635 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76813ee8-3124-42a9-bc0b-7179d4f408bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:00.793419 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76813ee8-3124-42a9-bc0b-7179d4f408bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:00.794879 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76813ee8-3124-42a9-bc0b-7179d4f408bc None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:00.809208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-76813ee8-3124-42a9-bc0b-7179d4f408bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:00.809208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:00.809208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:00.809208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:00.809208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:00.809208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:00.809208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:00.809208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:00.809208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:00.809208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:00.809208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:00.809208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:00.809208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:30:00.809208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:30:00.809208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:30:00.810111 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:30:00.810111 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:00.810111 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:00.810111 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:00.810111 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:00.810111 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:00.810111 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:00.810111 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:00.810111 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:00.810111 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:00.810111 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:00.810111 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:00.810111 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:00.810111 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:00.810111 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:00.810111 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:00.810111 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:00.811011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:30:00.811011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:30:00.811011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:30:00.811011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:30:00.811011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:30:00.811011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:30:00.811011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:00.811011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:00.811011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 416/823] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:00.811511 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76813ee8-3124-42a9-bc0b-7179d4f408bc None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:00.820999 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6e303c-49dc-4eb4-9225-0a11c49ea426 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:00.856633 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcddf96-0d63-4688-b28d-6f18341a6f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:00.857296 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcddf96-0d63-4688-b28d-6f18341a6f32 None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:00.857581 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcddf96-0d63-4688-b28d-6f18341a6f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:00.857859 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcddf96-0d63-4688-b28d-6f18341a6f32 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:00.858597 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcddf96-0d63-4688-b28d-6f18341a6f32 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:00.860041 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcddf96-0d63-4688-b28d-6f18341a6f32 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:00.872481 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:00.882393 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 408/824] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Jun 15 14:30:00 2022] POST /identity/v3/endpoints => generated 346 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:00.888777 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcddf96-0d63-4688-b28d-6f18341a6f32 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:00.937024 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76813ee8-3124-42a9-bc0b-7179d4f408bc None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:00.969493 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:00.997694 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60bcd6-deb9-4688-926e-4f8d7b08883e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:00.998364 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60bcd6-deb9-4688-926e-4f8d7b08883e None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:00.998647 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60bcd6-deb9-4688-926e-4f8d7b08883e None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:00.998921 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60bcd6-deb9-4688-926e-4f8d7b08883e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:01.036131 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcddf96-0d63-4688-b28d-6f18341a6f32 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:01.088368 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6835874-efe8-42c0-b3a2-3d0e0401c41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:01.088368 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6835874-efe8-42c0-b3a2-3d0e0401c41c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:01.088368 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6835874-efe8-42c0-b3a2-3d0e0401c41c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:01.088734 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6835874-efe8-42c0-b3a2-3d0e0401c41c None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:01.089410 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6835874-efe8-42c0-b3a2-3d0e0401c41c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:01.090266 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6835874-efe8-42c0-b3a2-3d0e0401c41c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:01.092542 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6835874-efe8-42c0-b3a2-3d0e0401c41c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:01.095253 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e60bcd6-deb9-4688-926e-4f8d7b08883e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:01.096635 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e60bcd6-deb9-4688-926e-4f8d7b08883e None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:01.108162 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 409/825] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:30:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:01.113525 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6835874-efe8-42c0-b3a2-3d0e0401c41c None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:01.155353 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:01.221412 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 417/826] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:00 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:30:01.222286 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6835874-efe8-42c0-b3a2-3d0e0401c41c None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:01.222809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60bcd6-deb9-4688-926e-4f8d7b08883e None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:01.237166 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:01.260279 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e81421-5b3c-4f7f-aed4-34fb6e268e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:01.261515 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e81421-5b3c-4f7f-aed4-34fb6e268e38 None admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:01.261948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e81421-5b3c-4f7f-aed4-34fb6e268e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:01.262370 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e81421-5b3c-4f7f-aed4-34fb6e268e38 None admin] PATH_INFO: `/v3/projects/d488c79cb1704e9496fc196d8aa6c104/users/bf393488726e49338e70017f5cb7c4be/roles/13dcf68cca9c44888c0fd27a78db3dbe` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:01.278890 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e81421-5b3c-4f7f-aed4-34fb6e268e38 None admin] RBAC: Authorizing `identity:create_grant(project_id=d488c79cb1704e9496fc196d8aa6c104, user_id=bf393488726e49338e70017f5cb7c4be, role_id=13dcf68cca9c44888c0fd27a78db3dbe)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:01.280564 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e81421-5b3c-4f7f-aed4-34fb6e268e38 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:01.297608 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 410/827] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Jun 15 14:30:01 2022] PUT /identity/v3/projects/d488c79cb1704e9496fc196d8aa6c104/users/bf393488726e49338e70017f5cb7c4be/roles/13dcf68cca9c44888c0fd27a78db3dbe => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:30:01.350047 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60bcd6-deb9-4688-926e-4f8d7b08883e None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:01.388577 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83329f01-30cf-4801-bcc0-fc8218e9dd37 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:01.389161 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83329f01-30cf-4801-bcc0-fc8218e9dd37 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:01.389987 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83329f01-30cf-4801-bcc0-fc8218e9dd37 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:01.392915 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 411/828] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:30:01.395628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab75eb-d011-4958-80cc-24195048475c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:01.396733 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab75eb-d011-4958-80cc-24195048475c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:01.397227 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab75eb-d011-4958-80cc-24195048475c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:01.397714 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab75eb-d011-4958-80cc-24195048475c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:01.398660 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59590054-31b7-47e2-8196-99d36703d502 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:01.398970 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59590054-31b7-47e2-8196-99d36703d502 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:01.399255 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59590054-31b7-47e2-8196-99d36703d502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:01.412323 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ab75eb-d011-4958-80cc-24195048475c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:01.413817 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ab75eb-d011-4958-80cc-24195048475c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:01.417764 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:01.425413 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-59590054-31b7-47e2-8196-99d36703d502 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:01.427774 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-59590054-31b7-47e2-8196-99d36703d502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:01.435628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-54ab75eb-d011-4958-80cc-24195048475c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:01.435628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:01.435628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:01.435628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:01.435628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:01.435628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:01.435628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:01.435628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:01.435628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:01.435628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:01.435628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:01.435628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:01.435628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:01.435628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:01.435628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:01.436576 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:01.436576 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:01.436576 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:01.436576 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:01.436576 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:01.436576 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:01.436576 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:01.436576 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:01.436576 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:01.436576 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:01.436576 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:01.436576 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:01.436576 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:01.436576 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:01.436576 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:01.436576 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:01.436576 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:01.437455 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:01.437455 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:01.437455 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:01.437455 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:01.437455 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 418/829] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:01.443044 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab75eb-d011-4958-80cc-24195048475c None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:01.524635 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 412/830] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:01 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:30:01.552351 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59590054-31b7-47e2-8196-99d36703d502 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:01.569425 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab75eb-d011-4958-80cc-24195048475c None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:01.607470 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0735111-eb2e-471a-a198-65b9aead0792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:01.608226 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0735111-eb2e-471a-a198-65b9aead0792 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:01.608471 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0735111-eb2e-471a-a198-65b9aead0792 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:01.608745 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0735111-eb2e-471a-a198-65b9aead0792 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:01.609517 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0735111-eb2e-471a-a198-65b9aead0792 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:01.609826 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0735111-eb2e-471a-a198-65b9aead0792 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:01.611242 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0735111-eb2e-471a-a198-65b9aead0792 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:01.623790 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 419/831] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:01.629296 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0735111-eb2e-471a-a198-65b9aead0792 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:01.672716 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:01.685210 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59590054-31b7-47e2-8196-99d36703d502 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:01.695364 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:01.738550 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f02240f-11c7-4563-a594-cbabbb99e414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:01.739530 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f02240f-11c7-4563-a594-cbabbb99e414 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:01.739910 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f02240f-11c7-4563-a594-cbabbb99e414 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:01.740289 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f02240f-11c7-4563-a594-cbabbb99e414 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:01.754045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0735111-eb2e-471a-a198-65b9aead0792 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:01.754962 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f02240f-11c7-4563-a594-cbabbb99e414 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:01.765675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-5f02240f-11c7-4563-a594-cbabbb99e414 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:01.765675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:01.765675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:01.765675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:01.765675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:01.765675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:01.765675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:01.765675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:01.765675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:01.765675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:01.765675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:01.765675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:01.765675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:01.765675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:01.765675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:01.765675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:01.766856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:01.766856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:01.766856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:01.766856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:01.766856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:01.766856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:01.766856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:01.766856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:01.766856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:01.766856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:01.766856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:01.766856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:01.766856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:01.766856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:01.766856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:01.766856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:01.767815 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:01.767815 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:01.767815 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:01.767815 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:01.767815 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:01.767815 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:01.767815 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:01.767815 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:01.767815 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:01.767815 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:01.767815 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:01.767815 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:01.771054 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 413/832] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:30:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:01.775115 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f02240f-11c7-4563-a594-cbabbb99e414 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:01.812098 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41eb072-8921-451e-8206-394a4e245609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:01.813448 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41eb072-8921-451e-8206-394a4e245609 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:01.813835 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41eb072-8921-451e-8206-394a4e245609 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:01.814470 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41eb072-8921-451e-8206-394a4e245609 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:01.823421 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41eb072-8921-451e-8206-394a4e245609 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:01.826039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41eb072-8921-451e-8206-394a4e245609 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:01.845972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-c41eb072-8921-451e-8206-394a4e245609 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:01.845972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:01.845972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:01.845972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:01.845972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:01.845972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:01.845972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:01.845972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:01.845972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:01.845972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:01.845972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:01.845972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:01.845972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:30:01.845972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:30:01.845972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:30:01.847109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:30:01.847109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:01.847109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:01.847109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:01.847109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:01.847109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:01.847109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:01.847109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:01.847109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:01.847109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:01.847109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:01.847109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:01.847109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:01.847109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:01.847109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:01.847109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:01.847109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:01.848237 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:30:01.848237 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:30:01.848237 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:30:01.848237 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:30:01.848237 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:30:01.848237 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:30:01.848237 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:01.848237 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:01.851027 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 420/833] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:01.856133 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41eb072-8921-451e-8206-394a4e245609 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:01.907683 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f02240f-11c7-4563-a594-cbabbb99e414 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:01.955086 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e5782-f8b6-4180-a799-ba0569c4df5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:01.956147 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e5782-f8b6-4180-a799-ba0569c4df5e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:01.957152 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e5782-f8b6-4180-a799-ba0569c4df5e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:01.957632 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e5782-f8b6-4180-a799-ba0569c4df5e None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:01.959054 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e5782-f8b6-4180-a799-ba0569c4df5e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:01.959569 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e5782-f8b6-4180-a799-ba0569c4df5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:01.961874 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e5782-f8b6-4180-a799-ba0569c4df5e None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:01.975163 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 414/834] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:01.982220 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e5782-f8b6-4180-a799-ba0569c4df5e None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:01.987890 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41eb072-8921-451e-8206-394a4e245609 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:02.004034 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:02.006380 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:02.028135 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328f853-97da-41b0-9554-991880eba126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:02.028818 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328f853-97da-41b0-9554-991880eba126 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:02.029179 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328f853-97da-41b0-9554-991880eba126 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:02.029527 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328f853-97da-41b0-9554-991880eba126 None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:02.030501 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2328f853-97da-41b0-9554-991880eba126 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:02.030901 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2328f853-97da-41b0-9554-991880eba126 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:02.032440 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2328f853-97da-41b0-9554-991880eba126 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:02.046090 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 421/835] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:30:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:02.052353 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328f853-97da-41b0-9554-991880eba126 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:02.115460 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e5782-f8b6-4180-a799-ba0569c4df5e None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:02.151603 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477fa20-d995-4ec9-a01f-11c09ad0a4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:02.152811 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477fa20-d995-4ec9-a01f-11c09ad0a4e6 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:02.153372 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477fa20-d995-4ec9-a01f-11c09ad0a4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:02.153652 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477fa20-d995-4ec9-a01f-11c09ad0a4e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:02.167363 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a477fa20-d995-4ec9-a01f-11c09ad0a4e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:02.169539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a477fa20-d995-4ec9-a01f-11c09ad0a4e6 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:02.173203 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328f853-97da-41b0-9554-991880eba126 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:02.191349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-a477fa20-d995-4ec9-a01f-11c09ad0a4e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:02.191349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:02.191349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:02.191349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:02.191349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:02.191349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:02.191349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:02.191349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:02.191349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:02.191349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:02.191349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:02.191349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:02.191349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:02.191349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:02.191349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:02.192269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:02.192269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:02.192269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:02.192269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:02.192269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:02.192269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:02.192269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:02.192269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:02.192269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:02.192269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:02.192269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:02.192269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:02.192269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:02.192269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:02.192269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:02.192269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:02.192269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:02.193129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:02.193129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:02.193129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:02.193129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:02.193511 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 415/836] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:02.199226 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477fa20-d995-4ec9-a01f-11c09ad0a4e6 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:02.214227 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6660ea5f-8fb3-47e5-9ad3-9ea2197ad233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:02.215114 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660ea5f-8fb3-47e5-9ad3-9ea2197ad233 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:02.215399 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660ea5f-8fb3-47e5-9ad3-9ea2197ad233 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:02.215809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660ea5f-8fb3-47e5-9ad3-9ea2197ad233 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:02.216652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6660ea5f-8fb3-47e5-9ad3-9ea2197ad233 None admin] RBAC: Adding query filter params (role.id=ed1cfa2fcf724a4d84f5acea100930c7, scope.project.id=d488c79cb1704e9496fc196d8aa6c104, user.id=f3185bd2458c4e22830975700888e9be) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:02.216963 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6660ea5f-8fb3-47e5-9ad3-9ea2197ad233 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:02.218501 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6660ea5f-8fb3-47e5-9ad3-9ea2197ad233 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:02.225477 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 422/837] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:02 2022] GET /identity/v3/role_assignments?user.id=f3185bd2458c4e22830975700888e9be&scope.project.id=d488c79cb1704e9496fc196d8aa6c104&role.id=ed1cfa2fcf724a4d84f5acea100930c7 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:02.268791 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:02.310525 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477fa20-d995-4ec9-a01f-11c09ad0a4e6 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:02.346198 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405042b-3f15-4149-b9d9-26a30c30c0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:02.346883 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405042b-3f15-4149-b9d9-26a30c30c0e8 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:02.347209 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405042b-3f15-4149-b9d9-26a30c30c0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:02.347491 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405042b-3f15-4149-b9d9-26a30c30c0e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:02.348228 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1405042b-3f15-4149-b9d9-26a30c30c0e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:02.348540 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1405042b-3f15-4149-b9d9-26a30c30c0e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:02.349934 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1405042b-3f15-4149-b9d9-26a30c30c0e8 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:02.368521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 416/838] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:02.370033 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6660ea5f-8fb3-47e5-9ad3-9ea2197ad233 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:02.417401 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:02.486159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6660ea5f-8fb3-47e5-9ad3-9ea2197ad233 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:02.522316 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d061fc-b6ae-452e-9dc6-22910d6b79fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:02.523160 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d061fc-b6ae-452e-9dc6-22910d6b79fa None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:02.523451 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d061fc-b6ae-452e-9dc6-22910d6b79fa None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:02.523730 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d061fc-b6ae-452e-9dc6-22910d6b79fa None admin] PATH_INFO: `/v3/users/swift` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:02.532363 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d061fc-b6ae-452e-9dc6-22910d6b79fa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:02.533817 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d061fc-b6ae-452e-9dc6-22910d6b79fa None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:02.549058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-45d061fc-b6ae-452e-9dc6-22910d6b79fa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:30:02.549058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:02.549058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:02.549058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:02.549058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:02.549058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:02.549058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:02.549058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:02.549058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:02.549058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:02.549058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:02.549058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:02.549058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:30:02.549058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:30:02.549058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:30:02.549058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:30:02.551217 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:02.551217 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:02.551217 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:02.551217 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:02.551217 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:02.551217 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:02.551217 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:02.551217 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:02.551217 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:02.551217 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:02.551217 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:02.551217 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:02.551217 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:02.551217 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:02.551217 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:02.551217 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:02.551217 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:02.551217 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:02.553377 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:02.553377 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:02.553377 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:30:02.553377 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:30:02.553377 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:30:02.553377 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:30:02.553377 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:30:02.553377 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:30:02.553377 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:30:02.553377 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:02.557934 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 423/839] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:30:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:02.558937 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405042b-3f15-4149-b9d9-26a30c30c0e8 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:02.669369 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405042b-3f15-4149-b9d9-26a30c30c0e8 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:02.696636 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:02.711312 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd8904-af09-4591-b8af-5829414dd57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:02.711997 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd8904-af09-4591-b8af-5829414dd57f None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:02.712283 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd8904-af09-4591-b8af-5829414dd57f None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:02.712563 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd8904-af09-4591-b8af-5829414dd57f None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:02.713408 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dd8904-af09-4591-b8af-5829414dd57f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:02.713727 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dd8904-af09-4591-b8af-5829414dd57f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:02.715721 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dd8904-af09-4591-b8af-5829414dd57f None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:02.735512 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 417/840] 10.222.0.45 () {56 vars in 1135 bytes} [Wed Jun 15 14:30:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:02.741106 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d061fc-b6ae-452e-9dc6-22910d6b79fa None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:02.865360 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:02.870882 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d061fc-b6ae-452e-9dc6-22910d6b79fa None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:02.915026 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152fa044-7fbf-466f-a410-392cf9371fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:02.915546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fa044-7fbf-466f-a410-392cf9371fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:02.919014 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fa044-7fbf-466f-a410-392cf9371fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:02.919014 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fa044-7fbf-466f-a410-392cf9371fb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:03.003106 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152fa044-7fbf-466f-a410-392cf9371fb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:03.003106 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152fa044-7fbf-466f-a410-392cf9371fb3 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:03.111544 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:03.114811 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 424/841] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:02 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:30:03.120438 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd8904-af09-4591-b8af-5829414dd57f None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:03.213971 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d2cce-c3c3-4af6-ba76-ca9125d4018b None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:03.214333 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d2cce-c3c3-4af6-ba76-ca9125d4018b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:03.215205 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d2cce-c3c3-4af6-ba76-ca9125d4018b None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:03.216842 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 425/842] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:03.222529 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2334dc6-3c7b-4627-8c75-42ad014d1aac None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:03.223096 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2334dc6-3c7b-4627-8c75-42ad014d1aac None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:03.223611 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2334dc6-3c7b-4627-8c75-42ad014d1aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:03.239856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd8904-af09-4591-b8af-5829414dd57f None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:03.258590 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-e2334dc6-3c7b-4627-8c75-42ad014d1aac None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:03.260171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e2334dc6-3c7b-4627-8c75-42ad014d1aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:03.321218 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be681dbe-bf83-4afb-9122-b941135ae8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:03.322417 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be681dbe-bf83-4afb-9122-b941135ae8ab None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:03.322898 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be681dbe-bf83-4afb-9122-b941135ae8ab None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:03.323393 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be681dbe-bf83-4afb-9122-b941135ae8ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:03.345312 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be681dbe-bf83-4afb-9122-b941135ae8ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:03.347196 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be681dbe-bf83-4afb-9122-b941135ae8ab None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:03.379432 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-be681dbe-bf83-4afb-9122-b941135ae8ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:03.379432 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:03.379432 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:03.379432 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:03.379432 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:03.379432 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:03.379432 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:03.379432 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:03.379432 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:03.379432 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:03.379432 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:03.379432 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:03.379432 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:03.379432 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:03.379432 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:03.380358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:03.380358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:03.380358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:03.380358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:03.380358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:03.380358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:03.380358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:03.380358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:03.380358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:03.380358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:03.380358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:03.380358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:03.380358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:03.380358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:03.380358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:03.380358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:03.380358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:03.381221 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:03.381221 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:03.381221 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:03.381221 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:03.390034 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 418/843] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:03.394152 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be681dbe-bf83-4afb-9122-b941135ae8ab None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:03.411111 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 426/844] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:03 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:03.451081 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2334dc6-3c7b-4627-8c75-42ad014d1aac None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:03.465350 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:03.537084 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be681dbe-bf83-4afb-9122-b941135ae8ab None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:03.567665 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2334dc6-3c7b-4627-8c75-42ad014d1aac None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:03.579587 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c35f7-7b25-4234-874f-7ed79733eb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:03.581000 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c35f7-7b25-4234-874f-7ed79733eb3c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:03.581482 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c35f7-7b25-4234-874f-7ed79733eb3c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:03.581950 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c35f7-7b25-4234-874f-7ed79733eb3c None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:03.582896 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c35f7-7b25-4234-874f-7ed79733eb3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:03.583451 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c35f7-7b25-4234-874f-7ed79733eb3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:03.585256 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c35f7-7b25-4234-874f-7ed79733eb3c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:03.602368 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 419/845] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:03.612254 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c35f7-7b25-4234-874f-7ed79733eb3c None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:03.719063 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:03.722932 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:03.729671 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c35f7-7b25-4234-874f-7ed79733eb3c None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:03.730463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea10091-983e-4033-ada6-7af57b378314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:03.731302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea10091-983e-4033-ada6-7af57b378314 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:03.731583 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea10091-983e-4033-ada6-7af57b378314 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:03.731856 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea10091-983e-4033-ada6-7af57b378314 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:03.732788 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea10091-983e-4033-ada6-7af57b378314 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:03.733970 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea10091-983e-4033-ada6-7af57b378314 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:03.745739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-2ea10091-983e-4033-ada6-7af57b378314 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:30:03.745739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:03.745739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:03.745739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:03.745739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:03.745739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:03.745739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:03.745739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:03.745739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:30:03.745739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:03.745739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:03.745739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:03.745739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:03.745739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:03.745739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:03.745739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:03.745739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:03.752148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:03.752148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:03.752148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:03.752148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:03.752148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:03.752148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:03.752148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:03.752148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:03.752148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:03.752148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:03.752148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:03.752148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:03.752148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:03.752148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:03.752148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:03.752148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:03.753268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:03.753268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:03.753268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:03.753268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:03.753268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:03.753268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:03.753268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:03.753268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:03.753268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:03.753268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:03.753268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:03.753268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:03.753268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:30:03.753268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:03.753268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 427/846] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Jun 15 14:30:03 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:03.754067 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac07af-b132-4209-9135-36ea6f62cb95 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:03.754067 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac07af-b132-4209-9135-36ea6f62cb95 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:03.754067 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac07af-b132-4209-9135-36ea6f62cb95 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:03.754067 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 428/847] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:03.755013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac07af-b132-4209-9135-36ea6f62cb95 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:03.773176 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9636ec3-86c4-4474-8822-9a93696e0fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:03.773964 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9636ec3-86c4-4474-8822-9a93696e0fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:03.774277 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9636ec3-86c4-4474-8822-9a93696e0fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:03.774585 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9636ec3-86c4-4474-8822-9a93696e0fd9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:03.782104 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9636ec3-86c4-4474-8822-9a93696e0fd9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:03.783560 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9636ec3-86c4-4474-8822-9a93696e0fd9 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:03.797638 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-e9636ec3-86c4-4474-8822-9a93696e0fd9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:03.797638 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:03.797638 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:03.797638 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:03.797638 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:03.797638 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:03.797638 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:03.797638 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:03.797638 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:03.797638 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:03.797638 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:03.797638 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:03.797638 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:30:03.797638 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:30:03.797638 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:30:03.798498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:30:03.798498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:03.798498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:03.798498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:03.798498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:03.798498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:03.798498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:03.798498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:03.798498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:03.798498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:03.798498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:03.798498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:03.798498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:03.798498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:03.798498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:03.798498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:03.798498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:03.799388 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:30:03.799388 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:30:03.799388 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:30:03.799388 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:30:03.799388 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:30:03.799388 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:30:03.799388 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:03.799388 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:03.799388 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 420/848] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:03.800291 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9de290-e726-4561-af78-1e10f1b7063e None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:03.800291 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9de290-e726-4561-af78-1e10f1b7063e None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:03.800529 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9de290-e726-4561-af78-1e10f1b7063e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:03.816723 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-3e9de290-e726-4561-af78-1e10f1b7063e None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:03.817888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-3e9de290-e726-4561-af78-1e10f1b7063e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:03.872586 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac07af-b132-4209-9135-36ea6f62cb95 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:03.897023 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 421/849] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:03 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:03.897679 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9de290-e726-4561-af78-1e10f1b7063e None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:03.919586 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc569a-98b3-44e4-8182-f53f0d8feccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:03.920580 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc569a-98b3-44e4-8182-f53f0d8feccf None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:03.921044 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc569a-98b3-44e4-8182-f53f0d8feccf None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:03.921383 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc569a-98b3-44e4-8182-f53f0d8feccf None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:03.922156 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdc569a-98b3-44e4-8182-f53f0d8feccf None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:03.922674 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdc569a-98b3-44e4-8182-f53f0d8feccf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:03.924236 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdc569a-98b3-44e4-8182-f53f0d8feccf None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:03.928987 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 429/850] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Jun 15 14:30:03 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:03.929944 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc569a-98b3-44e4-8182-f53f0d8feccf None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:03.988771 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:03.999540 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:04.015832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9de290-e726-4561-af78-1e10f1b7063e None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:04.034877 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc569a-98b3-44e4-8182-f53f0d8feccf None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:04.053295 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4579b56-18bc-4921-994e-d26b1ec60592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:04.053888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4579b56-18bc-4921-994e-d26b1ec60592 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:04.054172 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4579b56-18bc-4921-994e-d26b1ec60592 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:04.054444 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4579b56-18bc-4921-994e-d26b1ec60592 None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:04.055246 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4579b56-18bc-4921-994e-d26b1ec60592 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:04.055985 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4579b56-18bc-4921-994e-d26b1ec60592 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:04.057685 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4579b56-18bc-4921-994e-d26b1ec60592 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:04.069943 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d93d6-c8ad-43de-a8ba-d765333f3dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:04.070731 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d93d6-c8ad-43de-a8ba-d765333f3dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:04.071049 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d93d6-c8ad-43de-a8ba-d765333f3dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:04.071317 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d93d6-c8ad-43de-a8ba-d765333f3dd9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:04.079500 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 422/851] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:30:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:04.079500 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4579b56-18bc-4921-994e-d26b1ec60592 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:04.086126 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5d93d6-c8ad-43de-a8ba-d765333f3dd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:04.094169 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-0b5d93d6-c8ad-43de-a8ba-d765333f3dd9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:30:04.094169 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:04.094169 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:04.094169 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:04.094169 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:04.094169 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:04.094169 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:04.094169 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:04.094169 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:04.094169 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:04.094169 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:04.094169 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:04.094169 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:04.094169 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:04.094169 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:04.094169 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:04.100171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:04.100171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:04.100171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:04.100171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:04.100171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:04.100171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:04.100171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:04.100171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:04.100171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:04.100171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:04.100171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:04.100171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:04.100171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:04.100171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:04.100171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:04.100171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:04.101074 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:04.101074 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:04.101074 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:04.101074 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:04.101074 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:04.101074 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:04.101074 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:04.101074 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:04.101074 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:04.101074 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:04.101074 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:30:04.101074 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:04.101074 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 430/852] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Jun 15 14:30:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:04.101074 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d93d6-c8ad-43de-a8ba-d765333f3dd9 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:04.190625 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4579b56-18bc-4921-994e-d26b1ec60592 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:04.203803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d93d6-c8ad-43de-a8ba-d765333f3dd9 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:04.222423 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:04.225767 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b7fd4-8576-4ffa-a229-dd4cb5242145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:04.226429 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b7fd4-8576-4ffa-a229-dd4cb5242145 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:04.226717 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b7fd4-8576-4ffa-a229-dd4cb5242145 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:04.227025 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b7fd4-8576-4ffa-a229-dd4cb5242145 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:04.227726 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310b7fd4-8576-4ffa-a229-dd4cb5242145 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:04.228035 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310b7fd4-8576-4ffa-a229-dd4cb5242145 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:04.229398 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310b7fd4-8576-4ffa-a229-dd4cb5242145 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:04.235082 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 423/853] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Jun 15 14:30:04 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:04.235864 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b7fd4-8576-4ffa-a229-dd4cb5242145 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:04.251451 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bbe553-01c5-4524-9738-7bfc4b723693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:04.252126 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbe553-01c5-4524-9738-7bfc4b723693 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:04.252402 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbe553-01c5-4524-9738-7bfc4b723693 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:04.252683 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbe553-01c5-4524-9738-7bfc4b723693 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:04.253537 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bbe553-01c5-4524-9738-7bfc4b723693 None admin] RBAC: Adding query filter params (role.id=ed1cfa2fcf724a4d84f5acea100930c7, scope.project.id=d488c79cb1704e9496fc196d8aa6c104, user.id=641a81b3140f47119b5c2323a97a6681) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:04.253856 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bbe553-01c5-4524-9738-7bfc4b723693 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:04.255374 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bbe553-01c5-4524-9738-7bfc4b723693 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:04.256975 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:04.271512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 431/854] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:04 2022] GET /identity/v3/role_assignments?user.id=641a81b3140f47119b5c2323a97a6681&scope.project.id=d488c79cb1704e9496fc196d8aa6c104&role.id=ed1cfa2fcf724a4d84f5acea100930c7 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:04.366613 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b7fd4-8576-4ffa-a229-dd4cb5242145 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:04.412055 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed5c8b-141e-4255-b103-1bb634996bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:04.412948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed5c8b-141e-4255-b103-1bb634996bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:04.413240 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed5c8b-141e-4255-b103-1bb634996bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:04.413515 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed5c8b-141e-4255-b103-1bb634996bc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:04.414310 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed5c8b-141e-4255-b103-1bb634996bc7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:04.414615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed5c8b-141e-4255-b103-1bb634996bc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:04.416117 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed5c8b-141e-4255-b103-1bb634996bc7 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:04.427092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 424/855] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Jun 15 14:30:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:04.432623 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bbe553-01c5-4524-9738-7bfc4b723693 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:04.547229 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bbe553-01c5-4524-9738-7bfc4b723693 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:04.585190 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5096ea5-da13-4cd8-9895-055e288e3937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:04.586046 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5096ea5-da13-4cd8-9895-055e288e3937 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:04.586329 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5096ea5-da13-4cd8-9895-055e288e3937 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:04.586649 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5096ea5-da13-4cd8-9895-055e288e3937 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:04.598233 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5096ea5-da13-4cd8-9895-055e288e3937 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:04.599716 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5096ea5-da13-4cd8-9895-055e288e3937 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:04.606613 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff2a83-cf24-4370-879c-343cb489a85a None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:04.607097 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff2a83-cf24-4370-879c-343cb489a85a None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:04.607181 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff2a83-cf24-4370-879c-343cb489a85a None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:04.609060 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 425/856] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:30:04.614232 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422357f3-b264-4578-9b8c-f58d166040b3 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:04.614524 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422357f3-b264-4578-9b8c-f58d166040b3 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:04.614808 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422357f3-b264-4578-9b8c-f58d166040b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:04.624809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-d5096ea5-da13-4cd8-9895-055e288e3937 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:04.624809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:04.624809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:04.624809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:04.624809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:04.624809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:04.624809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:04.624809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:04.624809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:04.624809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:04.624809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:04.624809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:04.624809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:04.624809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:04.624809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:04.625712 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:04.625712 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:04.625712 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:04.625712 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:04.625712 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:04.625712 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:04.625712 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:04.625712 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:04.625712 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:04.625712 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:04.625712 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:04.625712 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:04.625712 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:04.625712 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:04.625712 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:04.625712 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:04.625712 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:04.626595 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:04.626595 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:04.626595 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:04.626595 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:04.626595 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 432/857] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:04.636872 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5096ea5-da13-4cd8-9895-055e288e3937 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:04.639245 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-422357f3-b264-4578-9b8c-f58d166040b3 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:04.640534 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-422357f3-b264-4578-9b8c-f58d166040b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:04.668985 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:04.723671 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 426/858] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:30:04.745212 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5096ea5-da13-4cd8-9895-055e288e3937 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:04.745794 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422357f3-b264-4578-9b8c-f58d166040b3 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:04.748246 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:04.790969 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcca7d0-032a-45ea-aa56-2334bdfcb0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:04.792254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcca7d0-032a-45ea-aa56-2334bdfcb0cb None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:04.792804 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcca7d0-032a-45ea-aa56-2334bdfcb0cb None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:04.793690 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcca7d0-032a-45ea-aa56-2334bdfcb0cb None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:04.794579 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcca7d0-032a-45ea-aa56-2334bdfcb0cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:04.795058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcca7d0-032a-45ea-aa56-2334bdfcb0cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:04.796963 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcca7d0-032a-45ea-aa56-2334bdfcb0cb None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:04.813965 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 433/859] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:04.823015 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcca7d0-032a-45ea-aa56-2334bdfcb0cb None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:04.861950 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422357f3-b264-4578-9b8c-f58d166040b3 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:04.906629 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e23bd-1944-4f59-8fc3-00393f165db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:04.907830 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e23bd-1944-4f59-8fc3-00393f165db9 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:04.908423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e23bd-1944-4f59-8fc3-00393f165db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:04.908999 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e23bd-1944-4f59-8fc3-00393f165db9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:04.909862 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77e23bd-1944-4f59-8fc3-00393f165db9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:04.911529 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77e23bd-1944-4f59-8fc3-00393f165db9 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:04.929238 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-b77e23bd-1944-4f59-8fc3-00393f165db9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:30:04.929238 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:04.929238 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:04.929238 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:04.929238 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:04.929238 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:04.929238 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:04.929238 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:04.929238 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:30:04.929238 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:04.929238 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:04.929238 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:04.929238 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:04.929238 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:04.929238 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:04.929238 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:04.930262 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:04.930262 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:04.930262 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:04.930262 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:04.930262 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:04.930262 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:04.930262 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:04.930262 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:04.930262 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:04.930262 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:04.930262 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:04.930262 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:04.930262 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:04.930262 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:04.930262 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:04.930262 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:04.931227 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:04.931227 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:04.931227 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:04.931227 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:04.931227 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:04.931227 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:04.931227 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:04.931227 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:04.931227 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:04.931227 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:04.931227 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:04.931227 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:04.931227 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:04.931227 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:30:04.931227 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:04.931227 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 427/860] 10.222.0.45 () {56 vars in 1132 bytes} [Wed Jun 15 14:30:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:04.940365 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e23bd-1944-4f59-8fc3-00393f165db9 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:04.949589 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:04.958726 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcca7d0-032a-45ea-aa56-2334bdfcb0cb None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:04.994779 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d972f27-898b-4896-bd93-2a2d1a2658fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:04.996276 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d972f27-898b-4896-bd93-2a2d1a2658fc None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:04.996637 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d972f27-898b-4896-bd93-2a2d1a2658fc None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:04.996980 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d972f27-898b-4896-bd93-2a2d1a2658fc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:05.004726 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d972f27-898b-4896-bd93-2a2d1a2658fc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:05.006278 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d972f27-898b-4896-bd93-2a2d1a2658fc None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:05.023833 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:05.030028 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-6d972f27-898b-4896-bd93-2a2d1a2658fc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:30:05.030028 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:05.030028 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:05.030028 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:05.030028 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:05.030028 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:05.030028 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:05.030028 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:05.030028 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:05.030028 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:05.030028 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:05.030028 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:05.030028 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:30:05.030028 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:30:05.030028 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:30:05.030028 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:30:05.030959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:05.030959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:05.030959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:05.030959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:05.030959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:05.030959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:05.030959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:05.030959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:05.030959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:05.030959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:05.030959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:05.030959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:05.030959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:05.030959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:05.030959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:05.030959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:05.030959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:05.030959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:05.032002 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:05.032002 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:05.032002 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:30:05.032002 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:30:05.032002 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:30:05.032002 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:30:05.032002 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:30:05.032002 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:30:05.032002 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:30:05.032002 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:05.032002 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 434/861] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Jun 15 14:30:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:05.037000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d972f27-898b-4896-bd93-2a2d1a2658fc None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:05.074667 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e23bd-1944-4f59-8fc3-00393f165db9 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:05.118939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6287d12-3b1c-475f-bcb1-fdc1e288757a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:05.120671 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6287d12-3b1c-475f-bcb1-fdc1e288757a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:05.121699 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6287d12-3b1c-475f-bcb1-fdc1e288757a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:05.122151 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6287d12-3b1c-475f-bcb1-fdc1e288757a None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:05.123521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6287d12-3b1c-475f-bcb1-fdc1e288757a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:05.124018 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6287d12-3b1c-475f-bcb1-fdc1e288757a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:05.126247 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6287d12-3b1c-475f-bcb1-fdc1e288757a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:05.138235 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 428/862] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Jun 15 14:30:04 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:05.138951 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6287d12-3b1c-475f-bcb1-fdc1e288757a None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:05.152618 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d972f27-898b-4896-bd93-2a2d1a2658fc None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:05.189447 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a3681-165e-4f4b-aad8-96ca1c5868d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:05.190038 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a3681-165e-4f4b-aad8-96ca1c5868d7 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:05.190318 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a3681-165e-4f4b-aad8-96ca1c5868d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:05.190590 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a3681-165e-4f4b-aad8-96ca1c5868d7 None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:05.191452 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a3681-165e-4f4b-aad8-96ca1c5868d7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:05.191762 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a3681-165e-4f4b-aad8-96ca1c5868d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:05.193198 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a3681-165e-4f4b-aad8-96ca1c5868d7 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:05.217609 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 435/863] 10.222.0.45 () {56 vars in 1149 bytes} [Wed Jun 15 14:30:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:05.218959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a3681-165e-4f4b-aad8-96ca1c5868d7 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:05.238461 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:05.277901 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6287d12-3b1c-475f-bcb1-fdc1e288757a None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:05.315408 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:05.320688 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8dabf6-9a65-4b51-a62c-f29ab556ca27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:05.321555 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8dabf6-9a65-4b51-a62c-f29ab556ca27 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:05.321919 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8dabf6-9a65-4b51-a62c-f29ab556ca27 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:05.322261 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8dabf6-9a65-4b51-a62c-f29ab556ca27 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:05.323345 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8dabf6-9a65-4b51-a62c-f29ab556ca27 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:05.323720 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8dabf6-9a65-4b51-a62c-f29ab556ca27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:05.325248 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8dabf6-9a65-4b51-a62c-f29ab556ca27 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:05.335025 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 429/864] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Jun 15 14:30:05 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:05.344152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a3681-165e-4f4b-aad8-96ca1c5868d7 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:05.383116 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58803e-a0da-43d8-8fee-e14adb04f966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:05.383781 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58803e-a0da-43d8-8fee-e14adb04f966 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:05.384064 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58803e-a0da-43d8-8fee-e14adb04f966 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:05.384336 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58803e-a0da-43d8-8fee-e14adb04f966 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:05.462694 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58803e-a0da-43d8-8fee-e14adb04f966 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:05.464175 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58803e-a0da-43d8-8fee-e14adb04f966 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:05.479666 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:05.579093 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 436/865] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:05 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:30:05.580520 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8dabf6-9a65-4b51-a62c-f29ab556ca27 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:05.683180 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8dabf6-9a65-4b51-a62c-f29ab556ca27 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:05.724104 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8bc382-1908-493f-9bf7-4e111644a031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:05.724828 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8bc382-1908-493f-9bf7-4e111644a031 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:05.725118 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8bc382-1908-493f-9bf7-4e111644a031 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:05.725402 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8bc382-1908-493f-9bf7-4e111644a031 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:05.741853 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8bc382-1908-493f-9bf7-4e111644a031 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:05.743407 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8bc382-1908-493f-9bf7-4e111644a031 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:05.760951 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-dd8bc382-1908-493f-9bf7-4e111644a031 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:05.760951 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:05.760951 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:05.760951 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:05.760951 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:05.760951 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:05.760951 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:05.760951 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:05.760951 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:05.760951 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:05.760951 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:05.760951 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:05.760951 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:05.760951 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:05.760951 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:05.761904 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:05.761904 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:05.761904 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:05.761904 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:05.761904 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:05.761904 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:05.761904 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:05.761904 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:05.761904 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:05.761904 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:05.761904 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:05.761904 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:05.761904 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:05.761904 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:05.761904 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:05.761904 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:05.761904 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:05.762785 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:05.762785 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:05.762785 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:05.762785 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:05.762785 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 430/866] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:05.767874 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58803e-a0da-43d8-8fee-e14adb04f966 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:05.869769 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58803e-a0da-43d8-8fee-e14adb04f966 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:05.883972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:05.908221 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a2dee-6f0d-4be5-8a63-28d6bb65ec99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:05.909216 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a2dee-6f0d-4be5-8a63-28d6bb65ec99 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:05.909701 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a2dee-6f0d-4be5-8a63-28d6bb65ec99 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:05.910173 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a2dee-6f0d-4be5-8a63-28d6bb65ec99 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:05.911152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4a2dee-6f0d-4be5-8a63-28d6bb65ec99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:05.911662 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4a2dee-6f0d-4be5-8a63-28d6bb65ec99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:05.913325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4a2dee-6f0d-4be5-8a63-28d6bb65ec99 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:05.925508 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 437/867] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:05.931043 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8bc382-1908-493f-9bf7-4e111644a031 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:05.983572 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:06.049418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8bc382-1908-493f-9bf7-4e111644a031 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:06.085482 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad968a7-3ac7-4594-ae8a-c1df23a6f8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:06.086893 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad968a7-3ac7-4594-ae8a-c1df23a6f8e9 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:06.087416 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad968a7-3ac7-4594-ae8a-c1df23a6f8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:06.087930 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad968a7-3ac7-4594-ae8a-c1df23a6f8e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:06.100635 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad968a7-3ac7-4594-ae8a-c1df23a6f8e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:06.102378 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad968a7-3ac7-4594-ae8a-c1df23a6f8e9 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:06.117465 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-4ad968a7-3ac7-4594-ae8a-c1df23a6f8e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:06.117465 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:06.117465 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:06.117465 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:06.117465 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:06.117465 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:06.117465 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:06.117465 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:06.117465 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:06.117465 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:06.117465 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:06.117465 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:06.117465 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:30:06.117465 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:30:06.117465 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:30:06.118391 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:30:06.118391 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:06.118391 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:06.118391 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:06.118391 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:06.118391 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:06.118391 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:06.118391 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:06.118391 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:06.118391 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:06.118391 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:06.118391 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:06.118391 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:06.118391 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:06.118391 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:06.118391 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:06.118391 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:06.119297 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:30:06.119297 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:30:06.119297 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:30:06.119297 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:30:06.119297 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:30:06.119297 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:30:06.119297 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:06.119297 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:06.121933 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 431/868] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:06.127354 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a2dee-6f0d-4be5-8a63-28d6bb65ec99 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:06.235007 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a2dee-6f0d-4be5-8a63-28d6bb65ec99 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:06.269317 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ac133-99ee-4cd9-81fb-eaf9baa5b9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:06.269976 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ac133-99ee-4cd9-81fb-eaf9baa5b9c3 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:06.270510 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ac133-99ee-4cd9-81fb-eaf9baa5b9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:06.270786 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ac133-99ee-4cd9-81fb-eaf9baa5b9c3 None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:06.271602 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ac133-99ee-4cd9-81fb-eaf9baa5b9c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:06.271911 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ac133-99ee-4cd9-81fb-eaf9baa5b9c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:06.273344 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ac133-99ee-4cd9-81fb-eaf9baa5b9c3 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:06.286156 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 438/869] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:30:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:06.292502 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad968a7-3ac7-4594-ae8a-c1df23a6f8e9 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:06.398896 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad968a7-3ac7-4594-ae8a-c1df23a6f8e9 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:06.407157 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:06.442121 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965ba46-1004-4a51-9b3f-6bb8ee2a4285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:06.442786 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965ba46-1004-4a51-9b3f-6bb8ee2a4285 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:06.443115 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965ba46-1004-4a51-9b3f-6bb8ee2a4285 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:06.443403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965ba46-1004-4a51-9b3f-6bb8ee2a4285 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:06.444246 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965ba46-1004-4a51-9b3f-6bb8ee2a4285 None admin] RBAC: Adding query filter params (role.id=13dcf68cca9c44888c0fd27a78db3dbe, scope.project.id=d488c79cb1704e9496fc196d8aa6c104, user.id=bf393488726e49338e70017f5cb7c4be) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:06.444700 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965ba46-1004-4a51-9b3f-6bb8ee2a4285 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:06.446147 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965ba46-1004-4a51-9b3f-6bb8ee2a4285 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:06.453584 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 432/870] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:06 2022] GET /identity/v3/role_assignments?user.id=bf393488726e49338e70017f5cb7c4be&scope.project.id=d488c79cb1704e9496fc196d8aa6c104&role.id=13dcf68cca9c44888c0fd27a78db3dbe => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:06.557105 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66cdb8-aeb9-4267-8320-0b9157c2fbcb None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:06.557443 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66cdb8-aeb9-4267-8320-0b9157c2fbcb None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:06.557669 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66cdb8-aeb9-4267-8320-0b9157c2fbcb None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:06.559990 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 439/871] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:06.565655 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb2755-278e-419e-8350-2a914c5111c3 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:06.565947 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb2755-278e-419e-8350-2a914c5111c3 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:06.566234 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb2755-278e-419e-8350-2a914c5111c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:06.584900 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-1cbb2755-278e-419e-8350-2a914c5111c3 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:06.586112 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1cbb2755-278e-419e-8350-2a914c5111c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:06.593002 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46c4cd-aad1-4e7b-94f3-11a31b7653be None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:06.593300 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46c4cd-aad1-4e7b-94f3-11a31b7653be None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:06.593555 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46c4cd-aad1-4e7b-94f3-11a31b7653be None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:06.594936 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 440/872] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:30:06.600744 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af1f14-b2ee-403c-b30d-b656e1d8176c None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:06.601039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af1f14-b2ee-403c-b30d-b656e1d8176c None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:06.601294 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af1f14-b2ee-403c-b30d-b656e1d8176c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:06.619568 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-69af1f14-b2ee-403c-b30d-b656e1d8176c None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:06.621521 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-69af1f14-b2ee-403c-b30d-b656e1d8176c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:06.653393 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:06.684505 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 433/873] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:06 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:06.700213 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb2755-278e-419e-8350-2a914c5111c3 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:06.716774 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 441/874] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:06 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:30:06.739941 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af1f14-b2ee-403c-b30d-b656e1d8176c None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:06.821400 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:06.825763 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb2755-278e-419e-8350-2a914c5111c3 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:06.865281 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5307ea1-57ba-43e4-9938-53bced5808c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:06.866207 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5307ea1-57ba-43e4-9938-53bced5808c0 None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:06.866592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5307ea1-57ba-43e4-9938-53bced5808c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:06.867392 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5307ea1-57ba-43e4-9938-53bced5808c0 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:06.867933 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5307ea1-57ba-43e4-9938-53bced5808c0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:06.869410 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5307ea1-57ba-43e4-9938-53bced5808c0 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:06.882175 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af1f14-b2ee-403c-b30d-b656e1d8176c None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:06.888173 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 434/875] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Jun 15 14:30:06 2022] POST /identity/v3/services => generated 251 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:30:06.934494 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948d9f3-91b4-4e79-b9c0-c2698f36073f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:06.935671 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948d9f3-91b4-4e79-b9c0-c2698f36073f None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:06.936104 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948d9f3-91b4-4e79-b9c0-c2698f36073f None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:06.936557 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948d9f3-91b4-4e79-b9c0-c2698f36073f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:06.937379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7948d9f3-91b4-4e79-b9c0-c2698f36073f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:06.939035 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7948d9f3-91b4-4e79-b9c0-c2698f36073f None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:06.952662 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-7948d9f3-91b4-4e79-b9c0-c2698f36073f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:30:06.952662 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:06.952662 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:06.952662 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:06.952662 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:06.952662 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:06.952662 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:06.952662 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:06.952662 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:30:06.952662 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:06.952662 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:06.952662 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:06.952662 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:06.952662 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:06.952662 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:06.952662 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:06.952662 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:06.953939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:06.953939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:06.953939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:06.953939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:06.953939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:06.953939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:06.953939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:06.953939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:06.953939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:06.953939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:06.953939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:06.953939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:06.953939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:06.953939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:06.953939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:06.953939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:06.955033 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:06.955033 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:06.955033 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:06.955033 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:06.955033 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:06.955033 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:06.955033 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:06.955033 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:06.955033 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:06.955033 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:06.955033 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:06.955033 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:06.955033 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:30:06.955033 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:06.957805 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 442/876] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Jun 15 14:30:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:06.967065 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5307ea1-57ba-43e4-9938-53bced5808c0 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:07.080220 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5307ea1-57ba-43e4-9938-53bced5808c0 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:07.118134 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0c3156-7247-4676-a4f7-e2651abeeef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:07.118817 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c3156-7247-4676-a4f7-e2651abeeef7 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:07.119136 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c3156-7247-4676-a4f7-e2651abeeef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:07.119419 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c3156-7247-4676-a4f7-e2651abeeef7 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:07.120138 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0c3156-7247-4676-a4f7-e2651abeeef7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:07.120449 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0c3156-7247-4676-a4f7-e2651abeeef7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:07.121896 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0c3156-7247-4676-a4f7-e2651abeeef7 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:07.130373 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 435/877] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Jun 15 14:30:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:07.136420 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948d9f3-91b4-4e79-b9c0-c2698f36073f None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:07.163795 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:07.251196 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948d9f3-91b4-4e79-b9c0-c2698f36073f None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:07.287477 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096b705-7a09-4f21-9b15-dc52c1fddd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:07.288581 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096b705-7a09-4f21-9b15-dc52c1fddd04 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:07.289068 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096b705-7a09-4f21-9b15-dc52c1fddd04 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:07.289543 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096b705-7a09-4f21-9b15-dc52c1fddd04 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:07.290479 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0096b705-7a09-4f21-9b15-dc52c1fddd04 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:07.291023 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0096b705-7a09-4f21-9b15-dc52c1fddd04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:07.292642 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0096b705-7a09-4f21-9b15-dc52c1fddd04 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:07.303660 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 443/878] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Jun 15 14:30:07 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:07.636992 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621552d-ced9-4cf0-ba29-70772e002ee3 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:07.637522 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621552d-ced9-4cf0-ba29-70772e002ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:07.637965 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621552d-ced9-4cf0-ba29-70772e002ee3 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:07.639749 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 436/879] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:07.645563 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30b53b-3fe5-48fc-9fd9-7bd6a9630279 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:07.646026 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30b53b-3fe5-48fc-9fd9-7bd6a9630279 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:07.646467 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30b53b-3fe5-48fc-9fd9-7bd6a9630279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:07.664794 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-2b30b53b-3fe5-48fc-9fd9-7bd6a9630279 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:07.666193 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-2b30b53b-3fe5-48fc-9fd9-7bd6a9630279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:07.736781 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 444/880] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:07 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:30:07.759697 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9621552d-ced9-4cf0-ba29-70772e002ee3 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:07.788532 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:07.872182 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9621552d-ced9-4cf0-ba29-70772e002ee3 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:07.912296 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d52e31-6a70-4ef2-83a3-cadcc31b5bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:07.912898 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d52e31-6a70-4ef2-83a3-cadcc31b5bc3 None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:07.913184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d52e31-6a70-4ef2-83a3-cadcc31b5bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:07.913464 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d52e31-6a70-4ef2-83a3-cadcc31b5bc3 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:07.914202 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d52e31-6a70-4ef2-83a3-cadcc31b5bc3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:07.915606 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d52e31-6a70-4ef2-83a3-cadcc31b5bc3 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:07.930591 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 437/881] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Jun 15 14:30:07 2022] POST /identity/v3/services => generated 269 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:08.860353 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d162394-d479-47b6-aca5-fd42fc53277b None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:08.860884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d162394-d479-47b6-aca5-fd42fc53277b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:08.860884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d162394-d479-47b6-aca5-fd42fc53277b None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:08.862556 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 445/882] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:08.867696 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153a3a5-8342-4f74-b4d3-14108243ec33 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:08.868071 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153a3a5-8342-4f74-b4d3-14108243ec33 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:08.868252 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153a3a5-8342-4f74-b4d3-14108243ec33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:08.885988 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-e153a3a5-8342-4f74-b4d3-14108243ec33 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:08.887766 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e153a3a5-8342-4f74-b4d3-14108243ec33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:08.969650 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 438/883] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:08 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:08.993489 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d162394-d479-47b6-aca5-fd42fc53277b None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:09.012747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:09.137626 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d162394-d479-47b6-aca5-fd42fc53277b None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:09.188740 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e47aa9-4f9d-4e59-9fc0-40dcfd84a96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:09.192707 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e47aa9-4f9d-4e59-9fc0-40dcfd84a96c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:09.192707 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e47aa9-4f9d-4e59-9fc0-40dcfd84a96c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:09.192707 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e47aa9-4f9d-4e59-9fc0-40dcfd84a96c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:09.215545 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e47aa9-4f9d-4e59-9fc0-40dcfd84a96c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:09.225206 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-41e47aa9-4f9d-4e59-9fc0-40dcfd84a96c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:30:09.225206 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:09.225206 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:09.225206 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:09.225206 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:09.225206 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:09.225206 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:09.225206 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:09.225206 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:09.225206 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:09.225206 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:09.225206 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:09.225206 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:09.225206 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:09.225206 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:09.225206 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:09.226221 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:09.226221 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:09.226221 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:09.226221 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:09.226221 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:09.226221 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:09.226221 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:09.226221 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:09.226221 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:09.226221 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:09.226221 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:09.226221 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:09.226221 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:09.226221 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:09.226221 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:09.226221 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:09.227151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:09.227151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:09.227151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:09.227151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:09.227151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:09.227151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:09.227151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:09.227151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:09.227151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:09.227151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:09.227151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:30:09.227151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:09.227151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 446/884] 10.222.0.45 () {56 vars in 1120 bytes} [Wed Jun 15 14:30:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:09.235247 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e153a3a5-8342-4f74-b4d3-14108243ec33 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:09.252038 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea600a3-3234-4e4a-a0aa-b27a875cd88b None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:09.252038 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea600a3-3234-4e4a-a0aa-b27a875cd88b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:09.252038 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea600a3-3234-4e4a-a0aa-b27a875cd88b None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:09.257216 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 447/885] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:09 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:09.259437 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:09.260597 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8f5b4-1682-46cc-9146-9e87421e1b7f None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:09.260842 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8f5b4-1682-46cc-9146-9e87421e1b7f None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:09.261216 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8f5b4-1682-46cc-9146-9e87421e1b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:09.289792 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-ceb8f5b4-1682-46cc-9146-9e87421e1b7f None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:09.291306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ceb8f5b4-1682-46cc-9146-9e87421e1b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:09.405923 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e153a3a5-8342-4f74-b4d3-14108243ec33 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:09.431250 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 448/886] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:09 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:30:09.454414 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb8f5b4-1682-46cc-9146-9e87421e1b7f None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:09.456925 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e48e63d-ad7e-4deb-b796-3a635c401e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:09.457612 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e48e63d-ad7e-4deb-b796-3a635c401e14 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:09.457901 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e48e63d-ad7e-4deb-b796-3a635c401e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:09.458178 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e48e63d-ad7e-4deb-b796-3a635c401e14 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:09.459247 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e48e63d-ad7e-4deb-b796-3a635c401e14 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:09.459562 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e48e63d-ad7e-4deb-b796-3a635c401e14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:09.461435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e48e63d-ad7e-4deb-b796-3a635c401e14 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:09.469555 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:09.473006 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 439/887] 10.222.0.45 () {56 vars in 1115 bytes} [Wed Jun 15 14:30:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:09.479621 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e48e63d-ad7e-4deb-b796-3a635c401e14 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:09.593092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:09.612181 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb8f5b4-1682-46cc-9146-9e87421e1b7f None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:09.641011 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e48e63d-ad7e-4deb-b796-3a635c401e14 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:09.656393 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee840f-3f87-4aec-9888-2b0f9a0a122c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:09.657191 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee840f-3f87-4aec-9888-2b0f9a0a122c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:09.657475 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee840f-3f87-4aec-9888-2b0f9a0a122c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:09.657754 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee840f-3f87-4aec-9888-2b0f9a0a122c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:09.658416 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bee840f-3f87-4aec-9888-2b0f9a0a122c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:09.659827 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bee840f-3f87-4aec-9888-2b0f9a0a122c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:09.671654 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-9bee840f-3f87-4aec-9888-2b0f9a0a122c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:30:09.671654 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:09.671654 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:09.671654 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:09.671654 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:09.671654 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:09.671654 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:09.671654 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:09.671654 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:30:09.671654 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:09.671654 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:09.671654 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:09.671654 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:09.671654 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:09.671654 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:09.671654 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:09.671654 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:09.672765 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:09.672765 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:09.672765 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:09.672765 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:09.672765 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:09.672765 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:09.672765 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:09.672765 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:09.672765 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:09.672765 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:09.672765 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:09.672765 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:09.672765 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:09.672765 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:09.672765 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:09.672765 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:09.673610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:09.673610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:09.673610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:09.673610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:09.673610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:09.673610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:09.673610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:09.673610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:09.673610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:09.673610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:09.673610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:09.673610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:09.673610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:30:09.673610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:09.673610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 449/888] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Jun 15 14:30:09 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:09.674415 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a3b84-afc1-45f6-be10-cc89140fccef None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:09.675111 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a3b84-afc1-45f6-be10-cc89140fccef None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:09.675111 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a3b84-afc1-45f6-be10-cc89140fccef None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:09.676348 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 450/889] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:09.679819 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a3b84-afc1-45f6-be10-cc89140fccef None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:09.686285 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8002361-f358-43ab-a013-b3b54afc5bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:09.687116 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8002361-f358-43ab-a013-b3b54afc5bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:09.687402 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8002361-f358-43ab-a013-b3b54afc5bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:09.687680 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8002361-f358-43ab-a013-b3b54afc5bb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:09.698662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8002361-f358-43ab-a013-b3b54afc5bb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:09.700681 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8002361-f358-43ab-a013-b3b54afc5bb0 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:09.719475 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-d8002361-f358-43ab-a013-b3b54afc5bb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:09.719475 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:09.719475 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:09.719475 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:09.719475 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:09.719475 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:09.719475 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:09.719475 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:09.719475 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:09.719475 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:09.719475 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:09.719475 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:09.719475 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:09.719475 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:09.719475 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:09.720405 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:09.720405 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:09.720405 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:09.720405 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:09.720405 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:09.720405 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:09.720405 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:09.720405 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:09.720405 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:09.720405 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:09.720405 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:09.720405 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:09.720405 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:09.720405 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:09.720405 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:09.720405 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:09.720405 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:09.721475 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:09.721475 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:09.721475 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:09.721475 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:09.728179 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 440/890] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:09.728179 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f8ce0-aacb-4a2e-95da-0a2a9ad4cd25 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:09.728179 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f8ce0-aacb-4a2e-95da-0a2a9ad4cd25 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:09.728179 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f8ce0-aacb-4a2e-95da-0a2a9ad4cd25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:09.748075 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-191f8ce0-aacb-4a2e-95da-0a2a9ad4cd25 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:09.749144 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-191f8ce0-aacb-4a2e-95da-0a2a9ad4cd25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:09.770845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:09.804723 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a3b84-afc1-45f6-be10-cc89140fccef None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:09.835020 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 441/891] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:09 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:09.835020 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f8ce0-aacb-4a2e-95da-0a2a9ad4cd25 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:09.845731 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e8ba46-e1b2-4c74-9fd5-0c623c927456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:09.846496 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8ba46-e1b2-4c74-9fd5-0c623c927456 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:09.846886 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8ba46-e1b2-4c74-9fd5-0c623c927456 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:09.847480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8ba46-e1b2-4c74-9fd5-0c623c927456 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:09.848070 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e8ba46-e1b2-4c74-9fd5-0c623c927456 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:09.848454 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e8ba46-e1b2-4c74-9fd5-0c623c927456 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:09.849912 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e8ba46-e1b2-4c74-9fd5-0c623c927456 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:09.856331 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 451/892] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Jun 15 14:30:09 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:09.857223 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e8ba46-e1b2-4c74-9fd5-0c623c927456 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:09.899701 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:09.962054 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f8ce0-aacb-4a2e-95da-0a2a9ad4cd25 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:09.971416 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e8ba46-e1b2-4c74-9fd5-0c623c927456 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:09.999865 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819e123-9a97-4b93-ae25-1775679bc047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:10.000526 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819e123-9a97-4b93-ae25-1775679bc047 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:10.000810 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819e123-9a97-4b93-ae25-1775679bc047 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:10.001086 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819e123-9a97-4b93-ae25-1775679bc047 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:10.001800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e819e123-9a97-4b93-ae25-1775679bc047 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:10.002108 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e819e123-9a97-4b93-ae25-1775679bc047 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:10.003524 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e819e123-9a97-4b93-ae25-1775679bc047 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:10.007221 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6976642f-1c91-48c9-8117-050166e41bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:10.007885 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976642f-1c91-48c9-8117-050166e41bc7 None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:10.008172 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976642f-1c91-48c9-8117-050166e41bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:10.008449 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976642f-1c91-48c9-8117-050166e41bc7 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:10.009176 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6976642f-1c91-48c9-8117-050166e41bc7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:10.010533 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6976642f-1c91-48c9-8117-050166e41bc7 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:10.012989 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:10.019186 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 442/893] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:10.020934 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819e123-9a97-4b93-ae25-1775679bc047 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:10.029707 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 452/894] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Jun 15 14:30:09 2022] POST /identity/v3/services => generated 241 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:10.036094 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6976642f-1c91-48c9-8117-050166e41bc7 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:10.148902 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819e123-9a97-4b93-ae25-1775679bc047 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:10.171289 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:10.219717 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a2f33-92d7-4249-b872-fca88dfbca43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:10.220389 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a2f33-92d7-4249-b872-fca88dfbca43 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:10.220676 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a2f33-92d7-4249-b872-fca88dfbca43 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:10.220953 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a2f33-92d7-4249-b872-fca88dfbca43 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:10.221662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05a2f33-92d7-4249-b872-fca88dfbca43 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:10.221973 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05a2f33-92d7-4249-b872-fca88dfbca43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:10.224088 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05a2f33-92d7-4249-b872-fca88dfbca43 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:10.229140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 443/895] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Jun 15 14:30:10 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:10.230317 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0cf48-58c8-4ba3-9e26-831eb533cc81 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:10.231175 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0cf48-58c8-4ba3-9e26-831eb533cc81 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:10.231457 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0cf48-58c8-4ba3-9e26-831eb533cc81 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:10.232850 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 444/896] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:10.236218 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d0cf48-58c8-4ba3-9e26-831eb533cc81 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:10.243898 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6976642f-1c91-48c9-8117-050166e41bc7 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:10.284469 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1ad43-c1a8-41f1-8d4c-e9e482cefb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:10.285674 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1ad43-c1a8-41f1-8d4c-e9e482cefb30 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:10.286188 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1ad43-c1a8-41f1-8d4c-e9e482cefb30 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:10.286658 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1ad43-c1a8-41f1-8d4c-e9e482cefb30 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:10.295127 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b1ad43-c1a8-41f1-8d4c-e9e482cefb30 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:10.296577 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b1ad43-c1a8-41f1-8d4c-e9e482cefb30 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:10.311121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-d6b1ad43-c1a8-41f1-8d4c-e9e482cefb30 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:30:10.311121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:10.311121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:10.311121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:10.311121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:10.311121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:10.311121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:10.311121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:10.311121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:10.311121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:10.311121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:10.311121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:10.311121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:30:10.311121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:30:10.311121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:30:10.311121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:30:10.312083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:10.312083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:10.312083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:10.312083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:10.312083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:10.312083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:10.312083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:10.312083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:10.312083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:10.312083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:10.312083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:10.312083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:10.312083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:10.312083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:10.312083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:10.312083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:10.312083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:10.312083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:10.313012 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:10.313012 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:10.313012 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:30:10.313012 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:30:10.313012 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:30:10.313012 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:30:10.313012 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:30:10.313012 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:30:10.313012 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:30:10.313012 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:10.313012 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 453/897] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Jun 15 14:30:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:10.313764 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224d093-2f13-42ce-a8b6-8bfe063ac505 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:10.314049 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224d093-2f13-42ce-a8b6-8bfe063ac505 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:10.314324 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224d093-2f13-42ce-a8b6-8bfe063ac505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:10.334791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-d224d093-2f13-42ce-a8b6-8bfe063ac505 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:10.336129 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d224d093-2f13-42ce-a8b6-8bfe063ac505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:10.354418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d0cf48-58c8-4ba3-9e26-831eb533cc81 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:10.392819 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:10.396382 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ced60c-bff9-4e1d-8658-df5e09f74bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:10.397103 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ced60c-bff9-4e1d-8658-df5e09f74bdb None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:10.397387 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ced60c-bff9-4e1d-8658-df5e09f74bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:10.397663 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ced60c-bff9-4e1d-8658-df5e09f74bdb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:10.398516 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ced60c-bff9-4e1d-8658-df5e09f74bdb None admin] RBAC: Adding query filter params (interface=public, service_id=6e97aaf674d24d449d89a1ac486d2b6f, region_id=RegionOne) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:10.398832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ced60c-bff9-4e1d-8658-df5e09f74bdb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:10.400397 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ced60c-bff9-4e1d-8658-df5e09f74bdb None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:10.405965 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 445/898] 10.222.0.45 () {56 vars in 1251 bytes} [Wed Jun 15 14:30:10 2022] GET /identity/v3/endpoints?service_id=6e97aaf674d24d449d89a1ac486d2b6f&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:10.406592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ced60c-bff9-4e1d-8658-df5e09f74bdb None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:10.412914 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 454/899] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:10.413606 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224d093-2f13-42ce-a8b6-8bfe063ac505 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:10.459228 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:10.525672 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ced60c-bff9-4e1d-8658-df5e09f74bdb None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:10.532966 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224d093-2f13-42ce-a8b6-8bfe063ac505 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:10.563808 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ed2c3-19c7-47c3-8eeb-fc189b8518d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:10.564478 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ed2c3-19c7-47c3-8eeb-fc189b8518d9 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:10.564762 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ed2c3-19c7-47c3-8eeb-fc189b8518d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:10.565034 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ed2c3-19c7-47c3-8eeb-fc189b8518d9 None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:10.565855 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730ed2c3-19c7-47c3-8eeb-fc189b8518d9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:10.566162 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730ed2c3-19c7-47c3-8eeb-fc189b8518d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:10.567635 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730ed2c3-19c7-47c3-8eeb-fc189b8518d9 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:10.572065 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23114a36-3146-47bc-926b-61eb99cf068b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:10.572977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23114a36-3146-47bc-926b-61eb99cf068b None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:10.573467 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23114a36-3146-47bc-926b-61eb99cf068b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:10.574555 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23114a36-3146-47bc-926b-61eb99cf068b None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:10.575760 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23114a36-3146-47bc-926b-61eb99cf068b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:10.577577 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23114a36-3146-47bc-926b-61eb99cf068b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:10.591029 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 446/900] 10.222.0.45 () {56 vars in 1149 bytes} [Wed Jun 15 14:30:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:10.591029 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ed2c3-19c7-47c3-8eeb-fc189b8518d9 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:10.591284 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 455/901] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Jun 15 14:30:10 2022] GET /identity/v3/services => generated 1497 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:30:10.592728 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23114a36-3146-47bc-926b-61eb99cf068b None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:10.669762 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:10.714157 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:10.721312 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ed2c3-19c7-47c3-8eeb-fc189b8518d9 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:10.746709 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23114a36-3146-47bc-926b-61eb99cf068b None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:10.765033 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a982399-de0f-431e-9c96-46704cb00acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:10.765849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a982399-de0f-431e-9c96-46704cb00acd None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:10.766184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a982399-de0f-431e-9c96-46704cb00acd None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:10.766434 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a982399-de0f-431e-9c96-46704cb00acd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:10.767077 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a982399-de0f-431e-9c96-46704cb00acd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:10.768706 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a982399-de0f-431e-9c96-46704cb00acd None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:10.782743 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-3a982399-de0f-431e-9c96-46704cb00acd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:30:10.782743 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:10.782743 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:10.782743 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:10.782743 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:10.782743 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:10.782743 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:10.782743 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:10.782743 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:30:10.782743 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:10.782743 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:10.782743 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:10.782743 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:10.782743 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:10.782743 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:10.782743 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:10.783713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:10.783713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:10.783713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:10.783713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:10.783713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:10.783713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:10.783713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:10.783713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:10.783713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:10.783713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:10.783713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:10.783713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:10.783713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:10.783713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:10.783713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:10.783713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:10.784714 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:10.784714 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:10.784714 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:10.784714 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:10.784714 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:10.784714 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:10.784714 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:10.784714 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:10.784714 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:10.784714 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:10.784714 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:10.784714 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:10.784714 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:10.784714 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:30:10.784714 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:10.784714 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 447/902] 10.222.0.45 () {56 vars in 1132 bytes} [Wed Jun 15 14:30:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:10.789832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a982399-de0f-431e-9c96-46704cb00acd None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:10.793110 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59789a56-95cb-4b99-a05c-61aad3b7648f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:10.794026 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59789a56-95cb-4b99-a05c-61aad3b7648f None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:10.794372 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59789a56-95cb-4b99-a05c-61aad3b7648f None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:10.795279 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59789a56-95cb-4b99-a05c-61aad3b7648f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:10.872589 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59789a56-95cb-4b99-a05c-61aad3b7648f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:10.874050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59789a56-95cb-4b99-a05c-61aad3b7648f None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:10.929803 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a982399-de0f-431e-9c96-46704cb00acd None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:10.938670 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:10.966609 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3575be91-0b70-4458-b520-d290cbafaf4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:10.967311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575be91-0b70-4458-b520-d290cbafaf4a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:10.967598 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575be91-0b70-4458-b520-d290cbafaf4a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:10.967880 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575be91-0b70-4458-b520-d290cbafaf4a None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:10.968690 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3575be91-0b70-4458-b520-d290cbafaf4a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:10.968896 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3575be91-0b70-4458-b520-d290cbafaf4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:10.970514 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3575be91-0b70-4458-b520-d290cbafaf4a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:10.980276 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 448/903] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Jun 15 14:30:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:10.982042 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3575be91-0b70-4458-b520-d290cbafaf4a None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:11.013899 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 456/904] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:10 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:30:11.015917 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59789a56-95cb-4b99-a05c-61aad3b7648f None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:11.016795 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:11.095449 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3575be91-0b70-4458-b520-d290cbafaf4a None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:11.127759 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59789a56-95cb-4b99-a05c-61aad3b7648f None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:11.136131 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61955dac-008c-4734-9448-0ff6b8b421ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:11.139501 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61955dac-008c-4734-9448-0ff6b8b421ba None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:11.141355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61955dac-008c-4734-9448-0ff6b8b421ba None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:11.141355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61955dac-008c-4734-9448-0ff6b8b421ba None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:11.141355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61955dac-008c-4734-9448-0ff6b8b421ba None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:11.141831 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61955dac-008c-4734-9448-0ff6b8b421ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:11.143724 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61955dac-008c-4734-9448-0ff6b8b421ba None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:11.151447 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 449/905] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Jun 15 14:30:10 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:11.156984 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61955dac-008c-4734-9448-0ff6b8b421ba None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:11.175979 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975fed16-d571-4ee5-bc6f-9afb57f3c9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:11.176849 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975fed16-d571-4ee5-bc6f-9afb57f3c9a1 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:11.177137 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975fed16-d571-4ee5-bc6f-9afb57f3c9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:11.177409 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975fed16-d571-4ee5-bc6f-9afb57f3c9a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:11.189885 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975fed16-d571-4ee5-bc6f-9afb57f3c9a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:11.191540 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975fed16-d571-4ee5-bc6f-9afb57f3c9a1 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:11.202333 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:11.212597 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-975fed16-d571-4ee5-bc6f-9afb57f3c9a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:11.212597 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:11.212597 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:11.212597 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:11.212597 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:11.212597 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:11.212597 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:11.212597 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:11.212597 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:11.212597 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:11.212597 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:11.212597 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:11.212597 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:11.212597 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:11.212597 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:11.213491 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:11.213491 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:11.213491 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:11.213491 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:11.213491 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:11.213491 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:11.213491 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:11.213491 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:11.213491 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:11.213491 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:11.213491 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:11.213491 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:11.213491 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:11.213491 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:11.213491 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:11.213491 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:11.213491 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:11.214371 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:11.214371 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:11.214371 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:11.214371 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:11.218758 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 457/906] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:11.220116 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975fed16-d571-4ee5-bc6f-9afb57f3c9a1 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:11.282874 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:11.298787 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61955dac-008c-4734-9448-0ff6b8b421ba None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:11.338451 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b768fdd6-234d-469a-94d7-90f082a2a1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:11.339701 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768fdd6-234d-469a-94d7-90f082a2a1ed None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:11.340193 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768fdd6-234d-469a-94d7-90f082a2a1ed None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:11.340674 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768fdd6-234d-469a-94d7-90f082a2a1ed None admin] PATH_INFO: `/v3/endpoints` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:11.341686 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b768fdd6-234d-469a-94d7-90f082a2a1ed None admin] RBAC: Adding query filter params (interface=public, service_id=e7a8f124054145bc8a0d628d72a29f10, region_id=RegionOne) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:11.342213 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b768fdd6-234d-469a-94d7-90f082a2a1ed None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:11.342388 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975fed16-d571-4ee5-bc6f-9afb57f3c9a1 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:11.344229 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b768fdd6-234d-469a-94d7-90f082a2a1ed None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:11.350558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 450/907] 10.222.0.45 () {56 vars in 1251 bytes} [Wed Jun 15 14:30:11 2022] GET /identity/v3/endpoints?service_id=e7a8f124054145bc8a0d628d72a29f10&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:11.359198 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b768fdd6-234d-469a-94d7-90f082a2a1ed None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:11.388002 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d110f33-d4e4-4b4d-b18b-ae88f55ac440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:11.388963 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d110f33-d4e4-4b4d-b18b-ae88f55ac440 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:11.389659 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d110f33-d4e4-4b4d-b18b-ae88f55ac440 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:11.389964 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d110f33-d4e4-4b4d-b18b-ae88f55ac440 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:11.390958 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d110f33-d4e4-4b4d-b18b-ae88f55ac440 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:11.391339 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d110f33-d4e4-4b4d-b18b-ae88f55ac440 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:11.393722 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d110f33-d4e4-4b4d-b18b-ae88f55ac440 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:11.410500 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 458/908] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:11.417030 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d110f33-d4e4-4b4d-b18b-ae88f55ac440 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:11.473547 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b768fdd6-234d-469a-94d7-90f082a2a1ed None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:11.510168 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a28763-a772-4133-bcd0-db870053f7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:11.510954 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a28763-a772-4133-bcd0-db870053f7bd None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:11.511354 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a28763-a772-4133-bcd0-db870053f7bd None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:11.513004 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a28763-a772-4133-bcd0-db870053f7bd None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:11.513004 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a28763-a772-4133-bcd0-db870053f7bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:11.513606 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:11.513900 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a28763-a772-4133-bcd0-db870053f7bd None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:11.520980 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 451/909] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Jun 15 14:30:11 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:30:11.561181 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d110f33-d4e4-4b4d-b18b-ae88f55ac440 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:11.604720 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec616515-d739-40f3-8462-438802b7308e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:11.606536 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec616515-d739-40f3-8462-438802b7308e None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:11.607051 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec616515-d739-40f3-8462-438802b7308e None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:11.607506 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec616515-d739-40f3-8462-438802b7308e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:11.615643 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec616515-d739-40f3-8462-438802b7308e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:11.617468 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec616515-d739-40f3-8462-438802b7308e None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:11.633671 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-ec616515-d739-40f3-8462-438802b7308e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:11.633671 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:11.633671 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:11.633671 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:11.633671 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:11.633671 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:11.633671 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:11.633671 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:11.633671 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:11.633671 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:11.633671 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:11.633671 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:11.633671 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:30:11.633671 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:30:11.633671 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:30:11.634661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:30:11.634661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:11.634661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:11.634661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:11.634661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:11.634661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:11.634661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:11.634661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:11.634661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:11.634661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:11.634661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:11.634661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:11.634661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:11.634661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:11.634661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:11.634661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:11.634661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:11.635615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:30:11.635615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:30:11.635615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:30:11.635615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:30:11.635615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:30:11.635615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:30:11.635615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:11.635615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:11.638136 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 459/910] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:11.643968 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a28763-a772-4133-bcd0-db870053f7bd None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:11.695593 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:11.772703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a28763-a772-4133-bcd0-db870053f7bd None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:11.812630 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec999332-10ec-4887-aca3-9cb53c4da198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:11.813687 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec999332-10ec-4887-aca3-9cb53c4da198 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:11.814188 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec999332-10ec-4887-aca3-9cb53c4da198 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:11.814662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec999332-10ec-4887-aca3-9cb53c4da198 None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:11.815732 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec999332-10ec-4887-aca3-9cb53c4da198 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:11.816269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec999332-10ec-4887-aca3-9cb53c4da198 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:11.817920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec999332-10ec-4887-aca3-9cb53c4da198 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:11.831687 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 452/911] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:30:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:11.840651 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec616515-d739-40f3-8462-438802b7308e None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:11.954025 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec616515-d739-40f3-8462-438802b7308e None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:11.976344 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:11.993134 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4245e-dd56-4caa-9a5e-ba13dfd47119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:11.993724 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4245e-dd56-4caa-9a5e-ba13dfd47119 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:11.994010 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4245e-dd56-4caa-9a5e-ba13dfd47119 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:11.994289 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4245e-dd56-4caa-9a5e-ba13dfd47119 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:11.995154 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4245e-dd56-4caa-9a5e-ba13dfd47119 None admin] RBAC: Adding query filter params (role.id=48dafbec535e4eee8263719557d59d6f, scope.project.id=d488c79cb1704e9496fc196d8aa6c104, user.id=bf393488726e49338e70017f5cb7c4be) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:11.995470 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4245e-dd56-4caa-9a5e-ba13dfd47119 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:11.997492 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4245e-dd56-4caa-9a5e-ba13dfd47119 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:12.005410 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 460/912] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:11 2022] GET /identity/v3/role_assignments?user.id=bf393488726e49338e70017f5cb7c4be&scope.project.id=d488c79cb1704e9496fc196d8aa6c104&role.id=48dafbec535e4eee8263719557d59d6f => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:12.282871 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48639eb-c29c-4e71-b33c-9ab675edbf31 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:12.283389 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48639eb-c29c-4e71-b33c-9ab675edbf31 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:12.283389 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48639eb-c29c-4e71-b33c-9ab675edbf31 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:12.285139 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 453/913] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:12.290332 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad13f3f-c9ff-4a6a-bbf0-3089a633561d None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:12.290750 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad13f3f-c9ff-4a6a-bbf0-3089a633561d None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:12.290890 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad13f3f-c9ff-4a6a-bbf0-3089a633561d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:12.313395 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-1ad13f3f-c9ff-4a6a-bbf0-3089a633561d None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:12.314592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1ad13f3f-c9ff-4a6a-bbf0-3089a633561d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:12.385645 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 461/914] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:12.408825 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48639eb-c29c-4e71-b33c-9ab675edbf31 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:12.511372 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:12.522351 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48639eb-c29c-4e71-b33c-9ab675edbf31 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:12.557225 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85609d2d-d4a0-4b77-be11-06205ad550b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:12.558035 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85609d2d-d4a0-4b77-be11-06205ad550b9 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:12.558853 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85609d2d-d4a0-4b77-be11-06205ad550b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:12.559164 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85609d2d-d4a0-4b77-be11-06205ad550b9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:12.559826 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85609d2d-d4a0-4b77-be11-06205ad550b9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:12.561196 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85609d2d-d4a0-4b77-be11-06205ad550b9 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:12.573605 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-85609d2d-d4a0-4b77-be11-06205ad550b9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:30:12.573605 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:12.573605 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:12.573605 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:12.573605 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:12.573605 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:12.573605 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:12.573605 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:12.573605 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:30:12.573605 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:12.573605 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:12.573605 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:12.573605 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:12.573605 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:12.573605 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:12.573605 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:12.573605 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:12.575188 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:12.575188 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:12.575188 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:12.575188 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:12.575188 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:12.575188 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:12.575188 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:12.575188 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:12.575188 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:12.575188 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:12.575188 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:12.575188 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:12.575188 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:12.575188 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:12.575188 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:12.575188 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:12.576015 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:12.576015 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:12.576015 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:12.576015 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:12.576015 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:12.576015 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:12.576015 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:12.576015 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:12.576015 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:12.576015 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:12.576015 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:12.576015 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:12.576015 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:30:12.576015 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:12.576015 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 454/915] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Jun 15 14:30:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:12.581675 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad13f3f-c9ff-4a6a-bbf0-3089a633561d None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:12.695528 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad13f3f-c9ff-4a6a-bbf0-3089a633561d None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:12.711284 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:12.739435 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b699dfd-32d0-4949-8980-de4a3c88614f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:12.740396 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b699dfd-32d0-4949-8980-de4a3c88614f None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:12.740847 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b699dfd-32d0-4949-8980-de4a3c88614f None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:12.741291 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b699dfd-32d0-4949-8980-de4a3c88614f None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:12.742165 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b699dfd-32d0-4949-8980-de4a3c88614f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:12.742636 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b699dfd-32d0-4949-8980-de4a3c88614f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:12.744206 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b699dfd-32d0-4949-8980-de4a3c88614f None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:12.750670 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 462/916] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Jun 15 14:30:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:12.756419 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85609d2d-d4a0-4b77-be11-06205ad550b9 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:12.856371 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76b3de-b140-4b72-acac-81b8fd40f8f1 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:12.856687 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76b3de-b140-4b72-acac-81b8fd40f8f1 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:12.856884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76b3de-b140-4b72-acac-81b8fd40f8f1 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:12.859085 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 463/917] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:30:12.863637 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6d1cc-058e-49d2-ae8f-edbaed761ecb None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:12.863927 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6d1cc-058e-49d2-ae8f-edbaed761ecb None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:12.864210 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6d1cc-058e-49d2-ae8f-edbaed761ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:12.877908 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85609d2d-d4a0-4b77-be11-06205ad550b9 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:12.882123 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-82a6d1cc-058e-49d2-ae8f-edbaed761ecb None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:12.883546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-82a6d1cc-058e-49d2-ae8f-edbaed761ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:12.912861 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685239bb-88b7-4644-b265-d90720da385b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:12.913561 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685239bb-88b7-4644-b265-d90720da385b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:12.913850 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685239bb-88b7-4644-b265-d90720da385b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:12.914227 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685239bb-88b7-4644-b265-d90720da385b None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:12.915031 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685239bb-88b7-4644-b265-d90720da385b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:12.915285 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685239bb-88b7-4644-b265-d90720da385b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:12.916662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685239bb-88b7-4644-b265-d90720da385b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:12.923282 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 455/918] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Jun 15 14:30:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:12.927843 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685239bb-88b7-4644-b265-d90720da385b None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:12.932380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:12.961877 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 464/919] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:30:12.985494 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a6d1cc-058e-49d2-ae8f-edbaed761ecb None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:13.061419 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685239bb-88b7-4644-b265-d90720da385b None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:13.093594 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a6d1cc-058e-49d2-ae8f-edbaed761ecb None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:13.103757 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:13.104283 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f55e7b-a88d-439b-8e59-eda4fb8f3d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:13.105187 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f55e7b-a88d-439b-8e59-eda4fb8f3d59 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:13.105697 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f55e7b-a88d-439b-8e59-eda4fb8f3d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:13.106166 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f55e7b-a88d-439b-8e59-eda4fb8f3d59 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:13.119438 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f55e7b-a88d-439b-8e59-eda4fb8f3d59 None admin] RBAC: Adding query filter params (interface=public, service_id=8967b7b30eef4daaaa96389256407d69, region_id=RegionOne) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:13.119438 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f55e7b-a88d-439b-8e59-eda4fb8f3d59 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:13.119438 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f55e7b-a88d-439b-8e59-eda4fb8f3d59 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:13.119855 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 456/920] 10.222.0.45 () {56 vars in 1251 bytes} [Wed Jun 15 14:30:12 2022] GET /identity/v3/endpoints?service_id=8967b7b30eef4daaaa96389256407d69&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:13.121470 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f55e7b-a88d-439b-8e59-eda4fb8f3d59 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:13.132652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee708f0-3717-4d46-a9d3-2184d6271e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:13.133460 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee708f0-3717-4d46-a9d3-2184d6271e6a None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:13.134000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee708f0-3717-4d46-a9d3-2184d6271e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:13.134000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee708f0-3717-4d46-a9d3-2184d6271e6a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:13.134669 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee708f0-3717-4d46-a9d3-2184d6271e6a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:13.136191 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee708f0-3717-4d46-a9d3-2184d6271e6a None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:13.150393 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-5ee708f0-3717-4d46-a9d3-2184d6271e6a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:30:13.150393 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:13.150393 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:13.150393 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:13.150393 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:13.150393 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:13.150393 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:13.150393 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:13.150393 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:30:13.150393 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:13.150393 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:13.150393 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:13.150393 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:13.150393 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:13.150393 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:13.150393 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:13.150393 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:13.151498 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:13.151498 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:13.151498 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:13.151498 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:13.151498 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:13.151498 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:13.151498 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:13.151498 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:13.151498 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:13.151498 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:13.151498 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:13.151498 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:13.151498 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:13.151498 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:13.151498 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:13.151498 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:13.152344 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:13.152344 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:13.152344 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:13.152344 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:13.152344 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:13.152344 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:13.152344 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:13.152344 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:13.152344 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:13.152344 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:13.152344 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:13.152344 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:13.152344 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:30:13.152344 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:13.158457 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 465/921] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Jun 15 14:30:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:13.160862 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee708f0-3717-4d46-a9d3-2184d6271e6a None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:13.200320 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:13.237398 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f55e7b-a88d-439b-8e59-eda4fb8f3d59 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:13.274767 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180d6b4-d69a-4936-9e8b-0ee3d5103879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:13.275850 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180d6b4-d69a-4936-9e8b-0ee3d5103879 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:13.276334 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180d6b4-d69a-4936-9e8b-0ee3d5103879 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:13.276802 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180d6b4-d69a-4936-9e8b-0ee3d5103879 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:13.277727 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180d6b4-d69a-4936-9e8b-0ee3d5103879 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:13.277967 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee708f0-3717-4d46-a9d3-2184d6271e6a None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:13.279544 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180d6b4-d69a-4936-9e8b-0ee3d5103879 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:13.295166 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 457/922] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Jun 15 14:30:13 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:30:13.320529 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54a191-f861-4137-850d-8ab87031e736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:13.323024 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54a191-f861-4137-850d-8ab87031e736 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:13.323024 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54a191-f861-4137-850d-8ab87031e736 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:13.323024 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54a191-f861-4137-850d-8ab87031e736 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:13.323024 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54a191-f861-4137-850d-8ab87031e736 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:13.323024 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54a191-f861-4137-850d-8ab87031e736 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:13.324873 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54a191-f861-4137-850d-8ab87031e736 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:13.335573 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 466/923] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Jun 15 14:30:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:13.340233 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180d6b4-d69a-4936-9e8b-0ee3d5103879 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:13.481269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180d6b4-d69a-4936-9e8b-0ee3d5103879 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:13.518405 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:13.522681 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793bbb86-a428-4187-ae1e-8f2c13b086fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:13.523447 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bbb86-a428-4187-ae1e-8f2c13b086fd None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:13.523734 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bbb86-a428-4187-ae1e-8f2c13b086fd None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:13.524014 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bbb86-a428-4187-ae1e-8f2c13b086fd None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:13.524731 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793bbb86-a428-4187-ae1e-8f2c13b086fd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:13.525049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793bbb86-a428-4187-ae1e-8f2c13b086fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:13.526437 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793bbb86-a428-4187-ae1e-8f2c13b086fd None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:13.535093 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 458/924] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Jun 15 14:30:13 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:13.537525 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54a191-f861-4137-850d-8ab87031e736 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:13.576396 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:13.651437 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54a191-f861-4137-850d-8ab87031e736 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:13.689012 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50143827-b8d9-4f87-8506-1a04aa91cac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:13.689684 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50143827-b8d9-4f87-8506-1a04aa91cac8 None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:13.689971 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50143827-b8d9-4f87-8506-1a04aa91cac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:13.690251 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50143827-b8d9-4f87-8506-1a04aa91cac8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:13.691020 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50143827-b8d9-4f87-8506-1a04aa91cac8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:13.692425 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50143827-b8d9-4f87-8506-1a04aa91cac8 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:13.731866 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 467/925] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Jun 15 14:30:13 2022] POST /identity/v3/endpoints => generated 346 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:13.868445 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc064a2d-4e0d-45b9-9298-660bdf37a5ee None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:13.868779 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc064a2d-4e0d-45b9-9298-660bdf37a5ee None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:13.870516 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc064a2d-4e0d-45b9-9298-660bdf37a5ee None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:13.871168 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 459/926] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:13.880953 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e662be-c1d3-4827-8fa7-b548f691589a None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:13.881254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e662be-c1d3-4827-8fa7-b548f691589a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:13.881529 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e662be-c1d3-4827-8fa7-b548f691589a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:13.917667 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-a9e662be-c1d3-4827-8fa7-b548f691589a None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:13.919912 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a9e662be-c1d3-4827-8fa7-b548f691589a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:14.007194 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 468/927] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:14.027650 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc064a2d-4e0d-45b9-9298-660bdf37a5ee None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:14.224388 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc064a2d-4e0d-45b9-9298-660bdf37a5ee None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:14.270000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f7229-5356-4b2c-ad41-8d6c7055fc81 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:14.270291 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f7229-5356-4b2c-ad41-8d6c7055fc81 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:14.270486 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f7229-5356-4b2c-ad41-8d6c7055fc81 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:14.272432 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 469/928] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:14.277972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac6632-9462-4224-aba8-50c1dcac7484 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:14.278495 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac6632-9462-4224-aba8-50c1dcac7484 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:14.278777 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac6632-9462-4224-aba8-50c1dcac7484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:14.297603 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-83ac6632-9462-4224-aba8-50c1dcac7484 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:14.298793 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-83ac6632-9462-4224-aba8-50c1dcac7484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:14.312818 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:14.360777 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febd6903-a535-4bc3-afb2-b8f0975832ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:14.362526 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd6903-a535-4bc3-afb2-b8f0975832ce None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:14.362637 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd6903-a535-4bc3-afb2-b8f0975832ce None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:14.362908 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd6903-a535-4bc3-afb2-b8f0975832ce None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:14.366781 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febd6903-a535-4bc3-afb2-b8f0975832ce None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:14.366781 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febd6903-a535-4bc3-afb2-b8f0975832ce None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:14.378539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-febd6903-a535-4bc3-afb2-b8f0975832ce None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:30:14.378539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:14.378539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:14.378539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:14.378539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:14.378539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:14.378539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:14.378539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:14.378539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:30:14.378539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:14.378539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:14.378539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:14.378539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:14.378539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:14.378539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:14.378539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:14.379489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:14.379489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:14.379489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:14.379489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:14.379489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:14.379489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:14.379489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:14.379489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:14.379489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:14.379489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:14.379489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:14.379489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:14.379489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:14.379489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:14.379489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:14.379489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:14.380311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:14.380311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:14.380311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:14.380311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:14.380311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:14.380311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:14.380311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:14.380311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:14.380311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:14.380311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:14.380311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:14.380311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:14.380311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:14.380311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:30:14.380311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:14.387074 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 460/929] 10.222.0.45 () {56 vars in 1132 bytes} [Wed Jun 15 14:30:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:14.387074 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febd6903-a535-4bc3-afb2-b8f0975832ce None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:14.480073 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:14.508223 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 470/930] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:14.535107 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febd6903-a535-4bc3-afb2-b8f0975832ce None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:14.536296 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ac6632-9462-4224-aba8-50c1dcac7484 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:14.574927 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838356c-6b40-4b86-bbe4-82e0f98f58b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:14.576012 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838356c-6b40-4b86-bbe4-82e0f98f58b5 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:14.576465 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838356c-6b40-4b86-bbe4-82e0f98f58b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:14.577265 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838356c-6b40-4b86-bbe4-82e0f98f58b5 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:14.578150 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a838356c-6b40-4b86-bbe4-82e0f98f58b5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:14.578633 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a838356c-6b40-4b86-bbe4-82e0f98f58b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:14.580185 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a838356c-6b40-4b86-bbe4-82e0f98f58b5 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:14.586042 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 461/931] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Jun 15 14:30:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:14.591905 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838356c-6b40-4b86-bbe4-82e0f98f58b5 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:14.709306 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838356c-6b40-4b86-bbe4-82e0f98f58b5 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:14.745522 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:14.757644 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae6c01a-ab53-494c-aec0-8156ac3d2920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:14.758574 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6c01a-ab53-494c-aec0-8156ac3d2920 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:14.759237 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6c01a-ab53-494c-aec0-8156ac3d2920 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:14.759625 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6c01a-ab53-494c-aec0-8156ac3d2920 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:14.760555 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae6c01a-ab53-494c-aec0-8156ac3d2920 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:14.761059 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae6c01a-ab53-494c-aec0-8156ac3d2920 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:14.763658 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae6c01a-ab53-494c-aec0-8156ac3d2920 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:14.774221 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 462/932] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Jun 15 14:30:14 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:14.775356 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae6c01a-ab53-494c-aec0-8156ac3d2920 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:14.890599 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae6c01a-ab53-494c-aec0-8156ac3d2920 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:14.911296 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ac6632-9462-4224-aba8-50c1dcac7484 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:14.928505 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a4aae-04cb-4ef7-9459-7078020c2bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:14.929483 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a4aae-04cb-4ef7-9459-7078020c2bf2 None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:14.929965 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a4aae-04cb-4ef7-9459-7078020c2bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:14.930462 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a4aae-04cb-4ef7-9459-7078020c2bf2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:14.931449 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27a4aae-04cb-4ef7-9459-7078020c2bf2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:14.933003 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27a4aae-04cb-4ef7-9459-7078020c2bf2 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:14.971466 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 463/933] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Jun 15 14:30:14 2022] POST /identity/v3/endpoints => generated 347 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:15.057099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcffb02-b0b2-4925-a341-18f5d5d463e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:15.058341 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcffb02-b0b2-4925-a341-18f5d5d463e2 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:15.058824 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcffb02-b0b2-4925-a341-18f5d5d463e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:15.059324 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcffb02-b0b2-4925-a341-18f5d5d463e2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:15.102891 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcffb02-b0b2-4925-a341-18f5d5d463e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:15.119959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-3fcffb02-b0b2-4925-a341-18f5d5d463e2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:30:15.119959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:15.119959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:15.119959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:15.119959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:15.119959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:15.119959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:15.119959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:15.119959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:15.119959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:15.119959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:15.119959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:15.119959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:15.119959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:15.119959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:15.119959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:15.120972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:15.120972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:15.120972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:15.120972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:15.120972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:15.120972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:15.120972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:15.120972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:15.120972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:15.120972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:15.120972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:15.120972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:15.120972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:15.120972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:15.120972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:15.120972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:15.121789 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:15.121789 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:15.121789 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:15.121789 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:15.121789 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:15.121789 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:15.121789 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:15.121789 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:15.121789 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:15.121789 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:15.121789 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:30:15.121789 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:15.133425 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 471/934] 10.222.0.45 () {56 vars in 1120 bytes} [Wed Jun 15 14:30:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 597 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:15.149707 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a4aae-04cb-4ef7-9459-7078020c2bf2 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:15.267522 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:15.336147 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a4aae-04cb-4ef7-9459-7078020c2bf2 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:15.406106 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1d13f-6b24-4b04-b832-ee6067b531a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:15.406709 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1d13f-6b24-4b04-b832-ee6067b531a3 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:15.411011 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1d13f-6b24-4b04-b832-ee6067b531a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:15.411011 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1d13f-6b24-4b04-b832-ee6067b531a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:15.411011 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1d13f-6b24-4b04-b832-ee6067b531a3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:15.411011 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1d13f-6b24-4b04-b832-ee6067b531a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:15.411318 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1d13f-6b24-4b04-b832-ee6067b531a3 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:15.422600 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 464/935] 10.222.0.45 () {56 vars in 1115 bytes} [Wed Jun 15 14:30:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:15.430308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcffb02-b0b2-4925-a341-18f5d5d463e2 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:15.452909 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:15.602482 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcffb02-b0b2-4925-a341-18f5d5d463e2 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:15.642175 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96017d7b-761b-401b-aa7f-7930abbde40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:15.642970 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96017d7b-761b-401b-aa7f-7930abbde40e None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:15.643281 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96017d7b-761b-401b-aa7f-7930abbde40e None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:15.643556 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96017d7b-761b-401b-aa7f-7930abbde40e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:15.655785 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96017d7b-761b-401b-aa7f-7930abbde40e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:15.657655 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96017d7b-761b-401b-aa7f-7930abbde40e None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:15.677968 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b8301-849d-4339-88e4-d466508e5614 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:15.678269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b8301-849d-4339-88e4-d466508e5614 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:15.678528 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b8301-849d-4339-88e4-d466508e5614 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:15.680177 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 465/936] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:30:15.684337 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-96017d7b-761b-401b-aa7f-7930abbde40e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:15.684337 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:15.684337 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:15.684337 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:15.684337 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:15.684337 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:15.684337 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:15.684337 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:15.684337 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:15.684337 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:15.684337 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:15.684337 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:15.684337 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:15.684337 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:15.684337 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:15.685204 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:15.685204 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:15.685204 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:15.685204 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:15.685204 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:15.685204 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:15.685204 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:15.685204 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:15.685204 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:15.685204 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:15.685204 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:15.685204 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:15.685204 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:15.685204 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:15.685204 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:15.685204 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:15.685204 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:15.686083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:15.686083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:15.686083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:15.686083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:15.686297 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc7e1d-bfbe-4d92-807e-6859737c5e03 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:15.686297 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc7e1d-bfbe-4d92-807e-6859737c5e03 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:15.686297 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc7e1d-bfbe-4d92-807e-6859737c5e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:15.688123 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 472/937] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:15.693383 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96017d7b-761b-401b-aa7f-7930abbde40e None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:15.708354 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-a7cc7e1d-bfbe-4d92-807e-6859737c5e03 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:15.709867 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-a7cc7e1d-bfbe-4d92-807e-6859737c5e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:15.758728 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:15.810594 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 466/938] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:15.813340 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96017d7b-761b-401b-aa7f-7930abbde40e None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:15.833218 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc7e1d-bfbe-4d92-807e-6859737c5e03 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:15.856259 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae139b16-a753-4a42-b335-e91786b51e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:15.857412 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae139b16-a753-4a42-b335-e91786b51e4e None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:15.857696 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae139b16-a753-4a42-b335-e91786b51e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:15.858269 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae139b16-a753-4a42-b335-e91786b51e4e None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:15.859018 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae139b16-a753-4a42-b335-e91786b51e4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:15.859511 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae139b16-a753-4a42-b335-e91786b51e4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:15.861455 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae139b16-a753-4a42-b335-e91786b51e4e None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:15.880382 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 473/939] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:15.881664 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:15.885874 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae139b16-a753-4a42-b335-e91786b51e4e None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:15.961362 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc7e1d-bfbe-4d92-807e-6859737c5e03 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:16.002226 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae139b16-a753-4a42-b335-e91786b51e4e None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:16.003986 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5841e7-0789-44e9-96c2-8a64e4788bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:16.005849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5841e7-0789-44e9-96c2-8a64e4788bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:16.006336 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5841e7-0789-44e9-96c2-8a64e4788bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:16.006812 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5841e7-0789-44e9-96c2-8a64e4788bb6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:16.007734 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5841e7-0789-44e9-96c2-8a64e4788bb6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:16.009318 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5841e7-0789-44e9-96c2-8a64e4788bb6 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:16.026461 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-dc5841e7-0789-44e9-96c2-8a64e4788bb6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:30:16.026461 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:16.026461 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:16.026461 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:16.026461 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:16.026461 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:16.026461 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:16.026461 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:16.026461 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:30:16.026461 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:16.026461 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:16.026461 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:16.026461 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:16.026461 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:16.026461 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:16.026461 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:16.026461 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:16.028288 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:16.028288 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:16.028288 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:16.028288 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:16.028288 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:16.028288 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:16.028288 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:16.028288 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:16.028288 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:16.028288 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:16.028288 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:16.028288 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:16.028288 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:16.028288 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:16.028288 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:16.028288 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:16.029092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:16.029092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:16.029092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:16.029092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:16.029092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:16.029092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:16.029092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:16.029092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:16.029092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:16.029092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:16.029092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:16.029092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:16.029092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:30:16.029092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:16.035327 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:16.037250 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 467/940] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Jun 15 14:30:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:16.038506 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5841e7-0789-44e9-96c2-8a64e4788bb6 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:16.048377 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07760e3-63bd-464c-829b-447f995191cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:16.050038 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07760e3-63bd-464c-829b-447f995191cb None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:16.050554 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07760e3-63bd-464c-829b-447f995191cb None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:16.051099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07760e3-63bd-464c-829b-447f995191cb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:16.062393 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07760e3-63bd-464c-829b-447f995191cb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:16.064136 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07760e3-63bd-464c-829b-447f995191cb None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:16.086748 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-d07760e3-63bd-464c-829b-447f995191cb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:30:16.086748 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:16.086748 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:16.086748 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:16.086748 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:16.086748 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:16.086748 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:16.086748 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:16.086748 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:16.086748 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:16.086748 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:16.086748 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:16.086748 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:30:16.086748 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:30:16.086748 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:30:16.086748 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:30:16.087884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:16.087884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:16.087884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:16.087884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:16.087884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:16.087884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:16.087884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:16.087884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:16.087884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:16.087884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:16.087884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:16.087884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:16.087884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:16.087884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:16.087884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:16.087884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:16.087884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:16.087884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:16.088803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:16.088803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:16.088803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:30:16.088803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:30:16.088803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:30:16.088803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:30:16.088803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:30:16.088803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:30:16.088803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:30:16.088803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:16.093946 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 474/941] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Jun 15 14:30:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:16.103054 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07760e3-63bd-464c-829b-447f995191cb None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:16.177168 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:16.181944 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5841e7-0789-44e9-96c2-8a64e4788bb6 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:16.215196 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07760e3-63bd-464c-829b-447f995191cb None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:16.225250 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b65c1-0dea-470b-b90f-3acd09983cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:16.225920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b65c1-0dea-470b-b90f-3acd09983cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:16.226197 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b65c1-0dea-470b-b90f-3acd09983cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:16.226470 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b65c1-0dea-470b-b90f-3acd09983cf8 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:16.227228 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b65c1-0dea-470b-b90f-3acd09983cf8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:16.227530 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b65c1-0dea-470b-b90f-3acd09983cf8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:16.228918 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b65c1-0dea-470b-b90f-3acd09983cf8 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:16.238075 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 468/942] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Jun 15 14:30:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:16.238075 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df3918-2c27-4a5f-a94e-f3ceb643ffc4 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:16.238075 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df3918-2c27-4a5f-a94e-f3ceb643ffc4 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:16.238075 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df3918-2c27-4a5f-a94e-f3ceb643ffc4 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:16.238075 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 469/943] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:16.240212 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df3918-2c27-4a5f-a94e-f3ceb643ffc4 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:16.259353 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4350e465-4ca5-4ca0-b0c9-bb2e848320dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:16.260454 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350e465-4ca5-4ca0-b0c9-bb2e848320dd None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:16.260853 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350e465-4ca5-4ca0-b0c9-bb2e848320dd None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:16.261201 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350e465-4ca5-4ca0-b0c9-bb2e848320dd None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:16.262754 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4350e465-4ca5-4ca0-b0c9-bb2e848320dd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:16.263515 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4350e465-4ca5-4ca0-b0c9-bb2e848320dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:16.265329 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4350e465-4ca5-4ca0-b0c9-bb2e848320dd None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:16.283914 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 475/944] 10.222.0.45 () {56 vars in 1149 bytes} [Wed Jun 15 14:30:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:16.285166 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2cc758-eca0-4de3-a199-87eb0653c444 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:16.285456 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2cc758-eca0-4de3-a199-87eb0653c444 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:16.285734 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2cc758-eca0-4de3-a199-87eb0653c444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:16.304835 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-da2cc758-eca0-4de3-a199-87eb0653c444 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:16.306079 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-da2cc758-eca0-4de3-a199-87eb0653c444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:16.338455 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:16.359032 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df3918-2c27-4a5f-a94e-f3ceb643ffc4 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:16.388485 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 476/945] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:16.391869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2cc758-eca0-4de3-a199-87eb0653c444 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:16.397424 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b5c81c-7f8c-4131-95e7-4570052a1c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:16.398087 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5c81c-7f8c-4131-95e7-4570052a1c19 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:16.398373 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5c81c-7f8c-4131-95e7-4570052a1c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:16.398700 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5c81c-7f8c-4131-95e7-4570052a1c19 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:16.399543 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b5c81c-7f8c-4131-95e7-4570052a1c19 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:16.399852 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b5c81c-7f8c-4131-95e7-4570052a1c19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:16.401285 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b5c81c-7f8c-4131-95e7-4570052a1c19 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:16.407904 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 470/946] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Jun 15 14:30:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:16.412071 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b5c81c-7f8c-4131-95e7-4570052a1c19 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:16.442057 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:16.506293 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2cc758-eca0-4de3-a199-87eb0653c444 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:16.526284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b5c81c-7f8c-4131-95e7-4570052a1c19 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:16.546231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224090e1-455a-4fc4-a7ec-d08b6bc774dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:16.547060 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224090e1-455a-4fc4-a7ec-d08b6bc774dd None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:16.547494 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224090e1-455a-4fc4-a7ec-d08b6bc774dd None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:16.547773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224090e1-455a-4fc4-a7ec-d08b6bc774dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:16.558584 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224090e1-455a-4fc4-a7ec-d08b6bc774dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:16.560079 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224090e1-455a-4fc4-a7ec-d08b6bc774dd None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:16.568607 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ddaa0d-6913-4266-89a1-04d3b719a269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:16.569271 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddaa0d-6913-4266-89a1-04d3b719a269 None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:16.569554 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddaa0d-6913-4266-89a1-04d3b719a269 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:16.569830 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddaa0d-6913-4266-89a1-04d3b719a269 None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:16.570521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ddaa0d-6913-4266-89a1-04d3b719a269 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:16.571944 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ddaa0d-6913-4266-89a1-04d3b719a269 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:16.573292 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: WARNING keystone.server.flask.common [None req-15ddaa0d-6913-4266-89a1-04d3b719a269 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 15 14:30:16.577950 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-224090e1-455a-4fc4-a7ec-d08b6bc774dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:16.577950 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:16.577950 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:16.577950 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:16.577950 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:16.577950 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:16.577950 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:16.577950 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:16.577950 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:16.577950 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:16.577950 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:16.577950 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:16.577950 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:16.577950 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:16.577950 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:16.578780 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:16.578780 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:16.578780 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:16.578780 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:16.578780 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:16.578780 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:16.578780 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:16.578780 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:16.578780 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:16.578780 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:16.578780 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:16.578780 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:16.578780 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:16.578780 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:16.578780 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:16.578780 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:16.578780 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:16.579946 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:16.579946 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:16.579946 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:16.579946 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:16.579946 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 477/947] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:16.579946 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224090e1-455a-4fc4-a7ec-d08b6bc774dd None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:16.592363 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.sql.core [None req-15ddaa0d-6913-4266-89a1-04d3b719a269 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 15 14:30:16.592363 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 15 14:30:16.592363 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [parameters: {'id': 'bbcf94bc7da34b1b8434c8ed5279024a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 15 14:30:16.592363 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80482) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 15 14:30:16.599388 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:16.608776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-15ddaa0d-6913-4266-89a1-04d3b719a269 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bbcf94bc7da34b1b8434c8ed5279024a.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bbcf94bc7da34b1b8434c8ed5279024a. Jun 15 14:30:16.608776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:16.608776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 14:30:16.608776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 14:30:16.608776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 14:30:16.608776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 14:30:16.608776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 14:30:16.608776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 14:30:16.608776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 14:30:16.608776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application conn.query(q) Jun 15 14:30:16.608776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 14:30:16.608776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 14:30:16.608776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 14:30:16.609600 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application result.read() Jun 15 14:30:16.609600 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 14:30:16.609600 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 14:30:16.609600 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 14:30:16.609600 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 14:30:16.609600 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 14:30:16.609600 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 14:30:16.609600 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 14:30:16.609600 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 14:30:16.609600 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 15 14:30:16.609600 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:16.609600 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 15 14:30:16.609600 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:16.609600 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:16.609600 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 15 14:30:16.609600 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 15 14:30:16.609600 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 15 14:30:16.609600 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 15 14:30:16.610521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 14:30:16.610521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application next(self.gen) Jun 15 14:30:16.610521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 15 14:30:16.610521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application yield resource Jun 15 14:30:16.610521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 14:30:16.610521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application next(self.gen) Jun 15 14:30:16.610521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 15 14:30:16.610521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self.session.rollback() Jun 15 14:30:16.610521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 15 14:30:16.610521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self.force_reraise() Jun 15 14:30:16.610521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 15 14:30:16.610521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise self.value Jun 15 14:30:16.610521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 15 14:30:16.610521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 15 14:30:16.610521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 15 14:30:16.610521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application session.commit() Jun 15 14:30:16.610521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 15 14:30:16.611435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 15 14:30:16.611435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 15 14:30:16.611435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._prepare_impl() Jun 15 14:30:16.611435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 15 14:30:16.611435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self.session.flush() Jun 15 14:30:16.611435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 15 14:30:16.611435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._flush(objects) Jun 15 14:30:16.611435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 15 14:30:16.611435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 15 14:30:16.611435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 15 14:30:16.611435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application compat.raise_( Jun 15 14:30:16.611435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 14:30:16.611435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception Jun 15 14:30:16.611435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 15 14:30:16.611435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application flush_context.execute() Jun 15 14:30:16.611435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 15 14:30:16.611435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rec.execute(self) Jun 15 14:30:16.612358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 15 14:30:16.612358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 15 14:30:16.612358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 15 14:30:16.612358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 15 14:30:16.612358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 15 14:30:16.612358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 15 14:30:16.612358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 15 14:30:16.612358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 15 14:30:16.612358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 15 14:30:16.612358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 15 14:30:16.612358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 15 14:30:16.612358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 15 14:30:16.612358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 15 14:30:16.612358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 15 14:30:16.612358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 15 14:30:16.613129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 15 14:30:16.613129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 14:30:16.613129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception Jun 15 14:30:16.613129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 14:30:16.613129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 14:30:16.613129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 14:30:16.613129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 14:30:16.613129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 14:30:16.613129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 14:30:16.613129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 14:30:16.613129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application conn.query(q) Jun 15 14:30:16.613129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 14:30:16.613129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 14:30:16.613129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 14:30:16.613129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application result.read() Jun 15 14:30:16.613129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 14:30:16.613129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 14:30:16.616180 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 14:30:16.616180 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 14:30:16.616180 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 14:30:16.616180 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 14:30:16.616180 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 14:30:16.616180 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 14:30:16.616180 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 15 14:30:16.616180 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 15 14:30:16.616180 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [parameters: {'id': 'bbcf94bc7da34b1b8434c8ed5279024a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 15 14:30:16.616180 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 15 14:30:16.616180 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:16.616180 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:16.616180 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:16.616180 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:16.616180 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 15 14:30:16.618799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 15 14:30:16.618799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 15 14:30:16.618799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 15 14:30:16.618799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 15 14:30:16.618799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:16.618799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:16.618799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:16.618799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:16.618799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:16.618799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:16.618799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:16.618799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:16.618799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:16.618799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:16.618799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:16.618799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:16.618799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:16.622304 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:16.622304 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 15 14:30:16.622304 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 15 14:30:16.622304 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:16.622304 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:16.622304 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 15 14:30:16.622304 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 15 14:30:16.622304 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bbcf94bc7da34b1b8434c8ed5279024a. Jun 15 14:30:16.622304 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:16.622304 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 471/948] 10.222.0.45 () {60 vars in 1140 bytes} [Wed Jun 15 14:30:16 2022] POST /identity/v3/projects => generated 262 bytes in 203 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:16.622304 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ddaa0d-6913-4266-89a1-04d3b719a269 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:16.692099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224090e1-455a-4fc4-a7ec-d08b6bc774dd None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:16.713322 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:16.729633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30f8e6-b8d1-4eee-9d37-9f652b3c00bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:16.730330 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30f8e6-b8d1-4eee-9d37-9f652b3c00bf None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:16.730613 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30f8e6-b8d1-4eee-9d37-9f652b3c00bf None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:16.730884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30f8e6-b8d1-4eee-9d37-9f652b3c00bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:16.731655 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30f8e6-b8d1-4eee-9d37-9f652b3c00bf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:16.733142 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30f8e6-b8d1-4eee-9d37-9f652b3c00bf None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:16.735417 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ddaa0d-6913-4266-89a1-04d3b719a269 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:16.757196 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 478/949] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Jun 15 14:30:16 2022] POST /identity/v3/endpoints => generated 349 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:16.758310 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30f8e6-b8d1-4eee-9d37-9f652b3c00bf None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:16.794351 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008720db-70f1-42ec-8b7e-3af1b978b377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:16.795125 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008720db-70f1-42ec-8b7e-3af1b978b377 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:16.795411 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008720db-70f1-42ec-8b7e-3af1b978b377 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:16.795839 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008720db-70f1-42ec-8b7e-3af1b978b377 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:16.796596 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008720db-70f1-42ec-8b7e-3af1b978b377 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:16.796902 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008720db-70f1-42ec-8b7e-3af1b978b377 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:16.798673 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008720db-70f1-42ec-8b7e-3af1b978b377 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:16.813398 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 472/950] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:16.818714 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008720db-70f1-42ec-8b7e-3af1b978b377 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:16.887471 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:16.932190 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30f8e6-b8d1-4eee-9d37-9f652b3c00bf None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:16.944939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008720db-70f1-42ec-8b7e-3af1b978b377 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:16.983246 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124323d-834c-4770-a34b-889137603b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:16.984105 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124323d-834c-4770-a34b-889137603b14 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:16.984331 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124323d-834c-4770-a34b-889137603b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:16.984606 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124323d-834c-4770-a34b-889137603b14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:16.986299 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7d70b-6e2c-4dee-9ec5-3b9a74bc0e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:16.987623 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7d70b-6e2c-4dee-9ec5-3b9a74bc0e3d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:16.987735 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7d70b-6e2c-4dee-9ec5-3b9a74bc0e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:16.987940 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7d70b-6e2c-4dee-9ec5-3b9a74bc0e3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:16.992676 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a124323d-834c-4770-a34b-889137603b14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:16.994144 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a124323d-834c-4770-a34b-889137603b14 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:16.996857 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc7d70b-6e2c-4dee-9ec5-3b9a74bc0e3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:16.997659 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc7d70b-6e2c-4dee-9ec5-3b9a74bc0e3d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:17.000899 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:17.011412 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-a124323d-834c-4770-a34b-889137603b14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:17.011412 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:17.011412 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:17.011412 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:17.011412 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:17.011412 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:17.011412 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:17.011412 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:17.011412 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:17.011412 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:17.011412 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:17.011412 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:17.011412 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:30:17.011412 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:30:17.011412 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:30:17.012884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:30:17.012884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:17.012884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:17.012884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:17.012884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:17.012884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:17.012884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:17.012884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:17.012884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:17.012884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:17.012884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:17.012884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:17.012884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:17.012884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:17.012884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:17.012884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:17.012884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:17.013749 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:30:17.013749 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:30:17.013749 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:30:17.013749 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:30:17.013749 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:30:17.013749 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:30:17.013749 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:17.013749 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:17.013749 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 479/951] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:17.021054 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124323d-834c-4770-a34b-889137603b14 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:17.021606 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-6fc7d70b-6e2c-4dee-9ec5-3b9a74bc0e3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:17.021606 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:17.021606 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:17.021606 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:17.021606 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:17.021606 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:17.021606 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:17.021606 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:17.021606 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:17.021606 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:17.021606 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:17.021606 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:17.021606 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:30:17.021606 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:30:17.021606 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:30:17.022655 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:30:17.022655 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:17.022655 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:17.022655 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:17.022655 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:17.022655 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:17.022655 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:17.022655 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:17.022655 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:17.022655 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:17.022655 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:17.022655 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:17.022655 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:17.022655 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:17.022655 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:17.022655 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:17.022655 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:17.023954 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:30:17.023954 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:30:17.023954 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:30:17.023954 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:30:17.023954 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:30:17.023954 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:30:17.023954 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:17.023954 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:17.026036 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 473/952] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:17.032320 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7d70b-6e2c-4dee-9ec5-3b9a74bc0e3d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:17.143155 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124323d-834c-4770-a34b-889137603b14 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:17.156013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7d70b-6e2c-4dee-9ec5-3b9a74bc0e3d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:17.176571 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:17.182893 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda4b2c-d971-45bf-9351-bbe43ce7f8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:17.184681 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda4b2c-d971-45bf-9351-bbe43ce7f8f8 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:17.184681 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda4b2c-d971-45bf-9351-bbe43ce7f8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:17.185902 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda4b2c-d971-45bf-9351-bbe43ce7f8f8 None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:17.186765 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cda4b2c-d971-45bf-9351-bbe43ce7f8f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:17.187198 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cda4b2c-d971-45bf-9351-bbe43ce7f8f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:17.188839 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cda4b2c-d971-45bf-9351-bbe43ce7f8f8 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:17.200247 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8bb9bc-b80b-47d9-bf86-bcde701a54d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:17.201187 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8bb9bc-b80b-47d9-bf86-bcde701a54d4 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:17.201646 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8bb9bc-b80b-47d9-bf86-bcde701a54d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:17.202095 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8bb9bc-b80b-47d9-bf86-bcde701a54d4 None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:17.203099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8bb9bc-b80b-47d9-bf86-bcde701a54d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:17.203722 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8bb9bc-b80b-47d9-bf86-bcde701a54d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:17.205316 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8bb9bc-b80b-47d9-bf86-bcde701a54d4 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:17.214191 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 480/953] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Jun 15 14:30:17 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:17.218662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 474/954] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:30:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:17.224961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda4b2c-d971-45bf-9351-bbe43ce7f8f8 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:17.356735 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda4b2c-d971-45bf-9351-bbe43ce7f8f8 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:17.385942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2f793-c577-4aeb-986e-e6368ceca651 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:17.386257 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2f793-c577-4aeb-986e-e6368ceca651 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:17.386540 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2f793-c577-4aeb-986e-e6368ceca651 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:17.388085 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 475/955] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:30:17.393949 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7efd5d5-2949-4e56-aa7e-754f2b46218e None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:17.394240 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7efd5d5-2949-4e56-aa7e-754f2b46218e None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:17.394548 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7efd5d5-2949-4e56-aa7e-754f2b46218e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:17.397167 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67fb5f3-c60b-42de-ac32-e578acf1b79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:17.398341 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67fb5f3-c60b-42de-ac32-e578acf1b79d None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:17.398827 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67fb5f3-c60b-42de-ac32-e578acf1b79d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:17.399350 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67fb5f3-c60b-42de-ac32-e578acf1b79d None admin] PATH_INFO: `/v3/projects/d488c79cb1704e9496fc196d8aa6c104/users/bf393488726e49338e70017f5cb7c4be/roles/48dafbec535e4eee8263719557d59d6f` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:17.401792 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67fb5f3-c60b-42de-ac32-e578acf1b79d None admin] RBAC: Authorizing `identity:create_grant(project_id=d488c79cb1704e9496fc196d8aa6c104, user_id=bf393488726e49338e70017f5cb7c4be, role_id=48dafbec535e4eee8263719557d59d6f)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:17.403597 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67fb5f3-c60b-42de-ac32-e578acf1b79d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:17.416451 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-b7efd5d5-2949-4e56-aa7e-754f2b46218e None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:17.418510 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b7efd5d5-2949-4e56-aa7e-754f2b46218e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:17.423048 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 481/956] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Jun 15 14:30:17 2022] PUT /identity/v3/projects/d488c79cb1704e9496fc196d8aa6c104/users/bf393488726e49338e70017f5cb7c4be/roles/48dafbec535e4eee8263719557d59d6f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:30:17.497625 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:17.507031 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 476/957] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:17.534098 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67fb5f3-c60b-42de-ac32-e578acf1b79d None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:17.604173 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:17.661656 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67fb5f3-c60b-42de-ac32-e578acf1b79d None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:17.697540 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb78b3b-3c30-47be-9d48-3827f07271f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:17.698340 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb78b3b-3c30-47be-9d48-3827f07271f7 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:17.698621 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb78b3b-3c30-47be-9d48-3827f07271f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:17.698894 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb78b3b-3c30-47be-9d48-3827f07271f7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:17.699619 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb78b3b-3c30-47be-9d48-3827f07271f7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:17.700982 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb78b3b-3c30-47be-9d48-3827f07271f7 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:17.712976 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-6fb78b3b-3c30-47be-9d48-3827f07271f7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:30:17.712976 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:17.712976 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:17.712976 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:17.712976 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:17.712976 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:17.712976 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:17.712976 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:17.712976 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:30:17.712976 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:17.712976 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:17.712976 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:17.712976 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:17.712976 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:17.712976 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:17.712976 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:17.712976 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:17.714103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:17.714103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:17.714103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:17.714103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:17.714103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:17.714103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:17.714103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:17.714103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:17.714103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:17.714103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:17.714103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:17.714103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:17.714103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:17.714103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:17.714103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:17.714103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:17.715586 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:17.715586 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:17.715586 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:17.715586 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:17.715586 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:17.715586 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:17.715586 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:17.715586 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:17.715586 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:17.715586 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:17.715586 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:17.715586 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:17.715586 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:30:17.715586 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:17.715586 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 482/958] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:17.722792 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7efd5d5-2949-4e56-aa7e-754f2b46218e None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:17.836171 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7efd5d5-2949-4e56-aa7e-754f2b46218e None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:17.881679 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b1a7c2-2b97-44b7-9f28-037b2cfa7569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:17.882350 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1a7c2-2b97-44b7-9f28-037b2cfa7569 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:17.882634 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1a7c2-2b97-44b7-9f28-037b2cfa7569 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:17.887029 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1a7c2-2b97-44b7-9f28-037b2cfa7569 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:17.887029 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b1a7c2-2b97-44b7-9f28-037b2cfa7569 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:17.887029 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b1a7c2-2b97-44b7-9f28-037b2cfa7569 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:17.887029 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b1a7c2-2b97-44b7-9f28-037b2cfa7569 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:17.891200 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 477/959] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Jun 15 14:30:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:17.899696 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb78b3b-3c30-47be-9d48-3827f07271f7 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:18.009523 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb78b3b-3c30-47be-9d48-3827f07271f7 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:18.028250 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:18.047499 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f4cf32-c74f-4b88-822a-0275288e3cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:18.048171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4cf32-c74f-4b88-822a-0275288e3cab None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:18.048454 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4cf32-c74f-4b88-822a-0275288e3cab None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:18.048726 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4cf32-c74f-4b88-822a-0275288e3cab None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:18.049437 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f4cf32-c74f-4b88-822a-0275288e3cab None admin] RBAC: Adding query filter params (type=compute) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:18.049749 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f4cf32-c74f-4b88-822a-0275288e3cab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:18.051276 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f4cf32-c74f-4b88-822a-0275288e3cab None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:18.056292 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 483/960] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Jun 15 14:30:17 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:19.019416 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18606fed-0368-4845-8b34-be1846b7fb62 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:19.019897 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18606fed-0368-4845-8b34-be1846b7fb62 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:19.019897 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18606fed-0368-4845-8b34-be1846b7fb62 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:19.021563 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 478/961] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:19.027762 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9a80d-404f-47c4-b265-01e2ce55a9ef None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:19.028193 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9a80d-404f-47c4-b265-01e2ce55a9ef None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:19.028272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9a80d-404f-47c4-b265-01e2ce55a9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:19.049126 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-31d9a80d-404f-47c4-b265-01e2ce55a9ef None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:19.050482 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-31d9a80d-404f-47c4-b265-01e2ce55a9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:19.121130 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 484/962] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:30:19.147634 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18606fed-0368-4845-8b34-be1846b7fb62 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:19.212914 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:19.262863 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18606fed-0368-4845-8b34-be1846b7fb62 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:19.300083 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ad8a0-7a55-42c2-a29f-ba70cdd26c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:19.300887 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ad8a0-7a55-42c2-a29f-ba70cdd26c02 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:19.301167 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ad8a0-7a55-42c2-a29f-ba70cdd26c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:19.301442 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ad8a0-7a55-42c2-a29f-ba70cdd26c02 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:19.302094 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85ad8a0-7a55-42c2-a29f-ba70cdd26c02 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:19.303486 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85ad8a0-7a55-42c2-a29f-ba70cdd26c02 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:19.316055 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-b85ad8a0-7a55-42c2-a29f-ba70cdd26c02 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:30:19.316055 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:19.316055 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:19.316055 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:19.316055 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:19.316055 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:19.316055 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:19.316055 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:19.316055 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:30:19.316055 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:19.316055 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:19.316055 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:19.316055 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:19.316055 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:19.316055 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:19.316055 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:19.316055 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:19.317094 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:19.317094 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:19.317094 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:19.317094 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:19.317094 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:19.317094 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:19.317094 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:19.317094 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:19.317094 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:19.317094 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:19.317094 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:19.317094 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:19.317094 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:19.317094 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:19.317094 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:19.317094 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:19.317920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:19.317920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:19.317920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:19.317920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:19.317920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:19.317920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:19.317920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:19.317920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:19.317920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:19.317920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:19.317920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:19.317920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:19.317920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:30:19.317920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:19.317920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 479/963] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Jun 15 14:30:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:19.323682 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9a80d-404f-47c4-b265-01e2ce55a9ef None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:19.442457 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9a80d-404f-47c4-b265-01e2ce55a9ef None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:19.468203 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:19.477075 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38debfaa-3c16-4040-ae0d-3e232e26926c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:19.477744 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38debfaa-3c16-4040-ae0d-3e232e26926c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:19.478028 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38debfaa-3c16-4040-ae0d-3e232e26926c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:19.478302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38debfaa-3c16-4040-ae0d-3e232e26926c None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:19.479056 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38debfaa-3c16-4040-ae0d-3e232e26926c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:19.479578 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38debfaa-3c16-4040-ae0d-3e232e26926c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:19.483377 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38debfaa-3c16-4040-ae0d-3e232e26926c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:19.489418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 485/964] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Jun 15 14:30:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:19.495012 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ad8a0-7a55-42c2-a29f-ba70cdd26c02 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:19.589098 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c035e-e264-4eed-a962-35827fbb839e None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:19.589416 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c035e-e264-4eed-a962-35827fbb839e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:19.589652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c035e-e264-4eed-a962-35827fbb839e None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:19.591267 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 486/965] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:19.599358 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712db00-1dea-4ebb-b644-2230a2b58324 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:19.599935 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712db00-1dea-4ebb-b644-2230a2b58324 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:19.600456 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712db00-1dea-4ebb-b644-2230a2b58324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:19.611057 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ad8a0-7a55-42c2-a29f-ba70cdd26c02 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:19.622773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-2712db00-1dea-4ebb-b644-2230a2b58324 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:19.624931 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-2712db00-1dea-4ebb-b644-2230a2b58324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:19.627668 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:19.650397 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea83a65-f767-4d20-b700-4911ba940242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:19.651291 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea83a65-f767-4d20-b700-4911ba940242 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:19.651673 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea83a65-f767-4d20-b700-4911ba940242 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:19.652088 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea83a65-f767-4d20-b700-4911ba940242 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:19.653007 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea83a65-f767-4d20-b700-4911ba940242 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:19.653416 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea83a65-f767-4d20-b700-4911ba940242 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:19.654931 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea83a65-f767-4d20-b700-4911ba940242 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:19.661418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 480/966] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Jun 15 14:30:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:19.667315 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea83a65-f767-4d20-b700-4911ba940242 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:19.726783 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 487/967] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:19.728186 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34868811-f6c6-4d81-ae90-927c4133090b None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:19.728474 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34868811-f6c6-4d81-ae90-927c4133090b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:19.728749 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34868811-f6c6-4d81-ae90-927c4133090b None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:19.730152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 488/968] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:19.735773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20943d5-3ea0-4c1a-b9a2-49500cee5dbf None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:19.736054 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20943d5-3ea0-4c1a-b9a2-49500cee5dbf None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:19.736438 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20943d5-3ea0-4c1a-b9a2-49500cee5dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:19.755261 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-d20943d5-3ea0-4c1a-b9a2-49500cee5dbf None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:19.756442 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d20943d5-3ea0-4c1a-b9a2-49500cee5dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:19.799794 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea83a65-f767-4d20-b700-4911ba940242 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:19.828568 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 489/969] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:30:19.829703 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20943d5-3ea0-4c1a-b9a2-49500cee5dbf None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:19.853955 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a62fe-8f5d-4807-9f00-08c5e66877a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:19.854589 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a62fe-8f5d-4807-9f00-08c5e66877a6 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:19.855067 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a62fe-8f5d-4807-9f00-08c5e66877a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:19.855269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a62fe-8f5d-4807-9f00-08c5e66877a6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:19.856647 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a62fe-8f5d-4807-9f00-08c5e66877a6 None admin] RBAC: Adding query filter params (interface=admin, service_id=8967b7b30eef4daaaa96389256407d69, region_id=RegionOne) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:19.856959 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a62fe-8f5d-4807-9f00-08c5e66877a6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:19.858316 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a62fe-8f5d-4807-9f00-08c5e66877a6 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:19.864036 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 481/970] 10.222.0.45 () {56 vars in 1249 bytes} [Wed Jun 15 14:30:19 2022] GET /identity/v3/endpoints?service_id=8967b7b30eef4daaaa96389256407d69&interface=admin®ion_id=RegionOne => generated 193 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:19.864946 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a62fe-8f5d-4807-9f00-08c5e66877a6 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:19.875078 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:19.916349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:19.951766 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20943d5-3ea0-4c1a-b9a2-49500cee5dbf None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:19.977483 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a62fe-8f5d-4807-9f00-08c5e66877a6 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:19.987823 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe2af6-ca39-4e07-89be-c79e7951b56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:19.988489 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe2af6-ca39-4e07-89be-c79e7951b56d None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:19.988769 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe2af6-ca39-4e07-89be-c79e7951b56d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:19.989042 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe2af6-ca39-4e07-89be-c79e7951b56d None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:19.989729 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fe2af6-ca39-4e07-89be-c79e7951b56d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:19.991194 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fe2af6-ca39-4e07-89be-c79e7951b56d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:19.992495 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: WARNING keystone.server.flask.common [None req-b2fe2af6-ca39-4e07-89be-c79e7951b56d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 15 14:30:20.005928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.sql.core [None req-b2fe2af6-ca39-4e07-89be-c79e7951b56d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 15 14:30:20.005928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 15 14:30:20.005928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [parameters: {'user_id': '4f1e753ede674fa0a873f0885c9cebc3', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 15 14:30:20.005928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80481) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 15 14:30:20.013266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-b2fe2af6-ca39-4e07-89be-c79e7951b56d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 15 14:30:20.013266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:20.013266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 14:30:20.013266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 14:30:20.013266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 14:30:20.013266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 14:30:20.013266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 14:30:20.013266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 14:30:20.013266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 14:30:20.013266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application conn.query(q) Jun 15 14:30:20.013266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 14:30:20.013266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 14:30:20.013266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 14:30:20.013266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application result.read() Jun 15 14:30:20.013266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 14:30:20.014099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 14:30:20.014099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 14:30:20.014099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 14:30:20.014099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 14:30:20.014099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 14:30:20.014099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 14:30:20.014099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 14:30:20.014099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 15 14:30:20.014099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:20.014099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 15 14:30:20.014099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:20.014099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:20.014099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 15 14:30:20.014099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 15 14:30:20.014099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 15 14:30:20.014099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 15 14:30:20.014099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 14:30:20.014099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application next(self.gen) Jun 15 14:30:20.015583 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 15 14:30:20.015583 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application yield resource Jun 15 14:30:20.015583 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 14:30:20.015583 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application next(self.gen) Jun 15 14:30:20.015583 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 15 14:30:20.015583 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self.session.rollback() Jun 15 14:30:20.015583 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 15 14:30:20.015583 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self.force_reraise() Jun 15 14:30:20.015583 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 15 14:30:20.015583 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise self.value Jun 15 14:30:20.015583 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 15 14:30:20.015583 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 15 14:30:20.015583 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 15 14:30:20.015583 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application session.commit() Jun 15 14:30:20.015583 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 15 14:30:20.015583 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 15 14:30:20.015583 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 15 14:30:20.016505 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._prepare_impl() Jun 15 14:30:20.016505 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 15 14:30:20.016505 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self.session.flush() Jun 15 14:30:20.016505 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 15 14:30:20.016505 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._flush(objects) Jun 15 14:30:20.016505 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 15 14:30:20.016505 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 15 14:30:20.016505 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 15 14:30:20.016505 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application compat.raise_( Jun 15 14:30:20.016505 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 14:30:20.016505 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception Jun 15 14:30:20.016505 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 15 14:30:20.016505 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application flush_context.execute() Jun 15 14:30:20.016505 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 15 14:30:20.016505 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rec.execute(self) Jun 15 14:30:20.016505 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 15 14:30:20.016505 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 15 14:30:20.017486 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 15 14:30:20.017486 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 15 14:30:20.017486 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 15 14:30:20.017486 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 15 14:30:20.017486 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 15 14:30:20.017486 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 15 14:30:20.017486 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 15 14:30:20.017486 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 15 14:30:20.017486 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 15 14:30:20.017486 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 15 14:30:20.017486 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 15 14:30:20.017486 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 15 14:30:20.017486 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 15 14:30:20.017486 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 15 14:30:20.017486 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 14:30:20.017486 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception Jun 15 14:30:20.018316 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 14:30:20.018316 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 14:30:20.018316 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 14:30:20.018316 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 14:30:20.018316 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 14:30:20.018316 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 14:30:20.018316 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 14:30:20.018316 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application conn.query(q) Jun 15 14:30:20.018316 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 14:30:20.018316 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 14:30:20.018316 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 14:30:20.018316 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application result.read() Jun 15 14:30:20.018316 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 14:30:20.018316 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 14:30:20.018316 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 14:30:20.018316 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 14:30:20.020418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 14:30:20.020418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 14:30:20.020418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 14:30:20.020418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 14:30:20.020418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 15 14:30:20.020418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 15 14:30:20.020418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [parameters: {'user_id': '4f1e753ede674fa0a873f0885c9cebc3', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 15 14:30:20.020418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 15 14:30:20.020418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:20.020418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:20.020418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:20.020418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:20.020418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:20.020418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:20.020418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:20.023034 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:20.023034 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:20.023034 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:20.023034 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:20.023034 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:20.023034 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:20.023034 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:20.023034 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 15 14:30:20.023034 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 15 14:30:20.023034 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:20.023034 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:20.023034 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:20.023034 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:20.023034 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:20.023034 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:20.023034 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 15 14:30:20.023034 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 15 14:30:20.028513 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 15 14:30:20.028513 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 15 14:30:20.028513 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 15 14:30:20.028513 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 15 14:30:20.028513 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 15 14:30:20.028513 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:20.028513 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 490/971] 10.222.0.45 () {60 vars in 1131 bytes} [Wed Jun 15 14:30:19 2022] POST /identity/v3/users => generated 160 bytes in 185 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:20.028513 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe2af6-ca39-4e07-89be-c79e7951b56d None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:20.029865 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72782022-09ce-4cea-865a-969d669f78a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:20.029865 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72782022-09ce-4cea-865a-969d669f78a9 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:20.029865 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72782022-09ce-4cea-865a-969d669f78a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:20.029865 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72782022-09ce-4cea-865a-969d669f78a9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:20.039421 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72782022-09ce-4cea-865a-969d669f78a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:20.048114 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-72782022-09ce-4cea-865a-969d669f78a9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:30:20.048114 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:20.048114 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:20.048114 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:20.048114 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:20.048114 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:20.048114 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:20.048114 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:20.048114 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:20.048114 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:20.048114 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:20.048114 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:20.048114 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:20.048114 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:20.048114 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:20.048114 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:20.051832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:20.051832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:20.051832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:20.051832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:20.051832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:20.051832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:20.051832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:20.051832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:20.051832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:20.051832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:20.051832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:20.051832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:20.051832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:20.051832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:20.051832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:20.051832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:20.052656 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:20.052656 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:20.052656 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:20.052656 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:20.052656 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:20.052656 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:20.052656 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:20.052656 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:20.052656 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:20.052656 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:20.052656 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:30:20.052656 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:20.052656 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 482/972] 10.222.0.45 () {56 vars in 1120 bytes} [Wed Jun 15 14:30:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:20.052656 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72782022-09ce-4cea-865a-969d669f78a9 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:20.122518 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:20.131051 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe2af6-ca39-4e07-89be-c79e7951b56d None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:20.161111 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72782022-09ce-4cea-865a-969d669f78a9 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:20.168920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:20.170306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f3a07-cd19-4cd4-ae34-046ba6678d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:20.170978 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f3a07-cd19-4cd4-ae34-046ba6678d67 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:20.171294 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f3a07-cd19-4cd4-ae34-046ba6678d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:20.171574 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f3a07-cd19-4cd4-ae34-046ba6678d67 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:20.172260 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49f3a07-cd19-4cd4-ae34-046ba6678d67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:20.173840 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49f3a07-cd19-4cd4-ae34-046ba6678d67 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:20.184618 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 491/973] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Jun 15 14:30:20 2022] GET /identity/v3/services => generated 1497 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:30:20.188101 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f3a07-cd19-4cd4-ae34-046ba6678d67 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:20.207194 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb44ce-8788-40db-8e85-2f3fab663546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:20.208560 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb44ce-8788-40db-8e85-2f3fab663546 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:20.209073 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb44ce-8788-40db-8e85-2f3fab663546 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:20.209561 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb44ce-8788-40db-8e85-2f3fab663546 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:20.219292 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efb44ce-8788-40db-8e85-2f3fab663546 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:20.221163 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efb44ce-8788-40db-8e85-2f3fab663546 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:20.239619 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-8efb44ce-8788-40db-8e85-2f3fab663546 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 14:30:20.239619 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:20.239619 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:20.239619 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:20.239619 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:20.239619 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:20.239619 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:20.239619 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:20.239619 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:20.239619 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:20.239619 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:20.239619 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:20.239619 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:30:20.239619 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:30:20.239619 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:30:20.239619 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:30:20.240700 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:20.240700 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:20.240700 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:20.240700 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:20.240700 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:20.240700 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:20.240700 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:20.240700 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:20.240700 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:20.240700 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:20.240700 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:20.240700 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:20.240700 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:20.240700 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:20.240700 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:20.240700 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:20.240700 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:20.240700 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:20.241615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:20.241615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:20.241615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:30:20.241615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:30:20.241615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:30:20.241615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:30:20.241615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:30:20.241615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:30:20.241615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 14:30:20.241615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:20.250253 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 483/974] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:20.251878 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb44ce-8788-40db-8e85-2f3fab663546 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:20.317132 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f3a07-cd19-4cd4-ae34-046ba6678d67 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:20.354878 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5550a8-471b-48b7-b8a1-13d9f84a526f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:20.356193 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5550a8-471b-48b7-b8a1-13d9f84a526f None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:20.356530 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5550a8-471b-48b7-b8a1-13d9f84a526f None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:20.356739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5550a8-471b-48b7-b8a1-13d9f84a526f None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:20.357942 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5550a8-471b-48b7-b8a1-13d9f84a526f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:20.358496 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5550a8-471b-48b7-b8a1-13d9f84a526f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:20.360865 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5550a8-471b-48b7-b8a1-13d9f84a526f None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:20.367971 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb44ce-8788-40db-8e85-2f3fab663546 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:20.371875 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 492/975] 10.222.0.45 () {56 vars in 1115 bytes} [Wed Jun 15 14:30:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:20.373831 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfada254-722c-4492-9055-94865ec32bc2 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:20.373831 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfada254-722c-4492-9055-94865ec32bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:20.373831 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfada254-722c-4492-9055-94865ec32bc2 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:20.374709 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 493/976] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:20.380142 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea786839-4b0a-45c6-af67-78e870bcbcc8 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:20.380436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea786839-4b0a-45c6-af67-78e870bcbcc8 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:20.380722 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea786839-4b0a-45c6-af67-78e870bcbcc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:20.398784 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-ea786839-4b0a-45c6-af67-78e870bcbcc8 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:20.400343 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ea786839-4b0a-45c6-af67-78e870bcbcc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:20.418397 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:20.418940 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974fc7b4-72a8-4863-9f13-8f8d059d1f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:20.419996 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974fc7b4-72a8-4863-9f13-8f8d059d1f9e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:20.419996 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974fc7b4-72a8-4863-9f13-8f8d059d1f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:20.420264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974fc7b4-72a8-4863-9f13-8f8d059d1f9e None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:20.421051 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974fc7b4-72a8-4863-9f13-8f8d059d1f9e None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:20.421362 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974fc7b4-72a8-4863-9f13-8f8d059d1f9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:20.422966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974fc7b4-72a8-4863-9f13-8f8d059d1f9e None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:20.462588 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 484/977] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Jun 15 14:30:20 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:20.463678 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974fc7b4-72a8-4863-9f13-8f8d059d1f9e None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:20.484367 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 494/978] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:20.501571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:20.507880 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea786839-4b0a-45c6-af67-78e870bcbcc8 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:20.618840 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974fc7b4-72a8-4863-9f13-8f8d059d1f9e None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:20.634560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea786839-4b0a-45c6-af67-78e870bcbcc8 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:20.666681 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae23d29-b1c2-45f7-8550-4fa9c0f7377f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:20.668434 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae23d29-b1c2-45f7-8550-4fa9c0f7377f None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:20.668913 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae23d29-b1c2-45f7-8550-4fa9c0f7377f None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:20.669352 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae23d29-b1c2-45f7-8550-4fa9c0f7377f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:20.680797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae23d29-b1c2-45f7-8550-4fa9c0f7377f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:20.682748 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae23d29-b1c2-45f7-8550-4fa9c0f7377f None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:20.688342 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90114473-e06b-49cc-a31f-c86b5bfa8e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:20.689038 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90114473-e06b-49cc-a31f-c86b5bfa8e08 None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:20.689325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90114473-e06b-49cc-a31f-c86b5bfa8e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:20.689601 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90114473-e06b-49cc-a31f-c86b5bfa8e08 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:20.690343 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90114473-e06b-49cc-a31f-c86b5bfa8e08 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:20.691756 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90114473-e06b-49cc-a31f-c86b5bfa8e08 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:20.709852 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-dae23d29-b1c2-45f7-8550-4fa9c0f7377f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:20.709852 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:20.709852 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:20.709852 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:20.709852 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:20.709852 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:20.709852 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:20.709852 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:20.709852 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:20.709852 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:20.709852 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:20.709852 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:20.709852 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:20.709852 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:20.709852 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:20.711558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:20.711558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:20.711558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:20.711558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:20.711558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:20.711558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:20.711558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:20.711558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:20.711558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:20.711558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:20.711558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:20.711558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:20.711558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:20.711558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:20.711558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:20.711558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:20.711558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:20.712730 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:20.712730 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:20.712730 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:20.712730 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:20.712948 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 495/980] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Jun 15 14:30:20 2022] POST /identity/v3/services => generated 242 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:20.716836 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 485/980] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:20.717882 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90114473-e06b-49cc-a31f-c86b5bfa8e08 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:20.742298 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:20.863366 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90114473-e06b-49cc-a31f-c86b5bfa8e08 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:20.901099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903c591-fb2e-4d63-be4d-87c142aa7612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:20.901701 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903c591-fb2e-4d63-be4d-87c142aa7612 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:20.901989 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903c591-fb2e-4d63-be4d-87c142aa7612 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:20.902813 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903c591-fb2e-4d63-be4d-87c142aa7612 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:20.903591 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8903c591-fb2e-4d63-be4d-87c142aa7612 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:20.903897 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8903c591-fb2e-4d63-be4d-87c142aa7612 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:20.905779 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8903c591-fb2e-4d63-be4d-87c142aa7612 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:20.919671 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 496/981] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:20.925453 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae23d29-b1c2-45f7-8550-4fa9c0f7377f None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:21.006629 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:21.038171 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae23d29-b1c2-45f7-8550-4fa9c0f7377f None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:21.073554 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3df32-cf6e-4dcf-a261-b9a64ee073cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:21.074485 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3df32-cf6e-4dcf-a261-b9a64ee073cd None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:21.074784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3df32-cf6e-4dcf-a261-b9a64ee073cd None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:21.075085 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3df32-cf6e-4dcf-a261-b9a64ee073cd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:21.082435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e3df32-cf6e-4dcf-a261-b9a64ee073cd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:21.083974 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e3df32-cf6e-4dcf-a261-b9a64ee073cd None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:21.099049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-03e3df32-cf6e-4dcf-a261-b9a64ee073cd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:30:21.099049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:21.099049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:21.099049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:21.099049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:21.099049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:21.099049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:21.099049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:21.099049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:21.099049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:21.099049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:21.099049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:21.099049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:30:21.099049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:30:21.099049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:30:21.099049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:30:21.099966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:21.099966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:21.099966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:21.099966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:21.099966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:21.099966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:21.099966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:21.099966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:21.099966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:21.099966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:21.099966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:21.099966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:21.099966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:21.099966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:21.099966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:21.099966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:21.099966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:21.099966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:21.101280 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:21.101280 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:21.101280 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:30:21.101280 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:30:21.101280 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:30:21.101280 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:30:21.101280 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:30:21.101280 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:30:21.101280 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:30:21.101280 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:21.102213 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 486/982] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Jun 15 14:30:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:21.106779 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903c591-fb2e-4d63-be4d-87c142aa7612 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:21.179389 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:21.214348 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903c591-fb2e-4d63-be4d-87c142aa7612 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:21.251104 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c55dcd-e017-4534-9b11-e574fe395f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:21.251804 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c55dcd-e017-4534-9b11-e574fe395f3e None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:21.252085 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c55dcd-e017-4534-9b11-e574fe395f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:21.252359 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c55dcd-e017-4534-9b11-e574fe395f3e None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:21.253192 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c55dcd-e017-4534-9b11-e574fe395f3e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:21.253498 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c55dcd-e017-4534-9b11-e574fe395f3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:21.254922 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c55dcd-e017-4534-9b11-e574fe395f3e None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:21.272551 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 497/983] 10.222.0.45 () {56 vars in 1149 bytes} [Wed Jun 15 14:30:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:21.278528 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3df32-cf6e-4dcf-a261-b9a64ee073cd None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:21.386510 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3df32-cf6e-4dcf-a261-b9a64ee073cd None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:21.422832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a9227-0b7d-45b7-89e7-22cec489b597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:21.423519 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a9227-0b7d-45b7-89e7-22cec489b597 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:21.423805 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a9227-0b7d-45b7-89e7-22cec489b597 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:21.424106 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a9227-0b7d-45b7-89e7-22cec489b597 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:21.428390 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:21.500014 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574a9227-0b7d-45b7-89e7-22cec489b597 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:21.501388 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574a9227-0b7d-45b7-89e7-22cec489b597 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:21.603076 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 487/984] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:21 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:30:21.609065 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c55dcd-e017-4534-9b11-e574fe395f3e None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:21.715281 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c55dcd-e017-4534-9b11-e574fe395f3e None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:21.758407 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:21.772893 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e124a82-365b-41c4-9daf-5488dde5a6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:21.773689 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e124a82-365b-41c4-9daf-5488dde5a6e9 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:21.773974 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e124a82-365b-41c4-9daf-5488dde5a6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:21.774254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e124a82-365b-41c4-9daf-5488dde5a6e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:21.785146 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e124a82-365b-41c4-9daf-5488dde5a6e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:21.786610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e124a82-365b-41c4-9daf-5488dde5a6e9 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:21.804854 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-8e124a82-365b-41c4-9daf-5488dde5a6e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:21.804854 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:21.804854 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:21.804854 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:21.804854 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:21.804854 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:21.804854 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:21.804854 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:21.804854 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:21.804854 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:21.804854 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:21.804854 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:21.804854 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:21.804854 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:21.804854 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:21.805803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:21.805803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:21.805803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:21.805803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:21.805803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:21.805803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:21.805803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:21.805803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:21.805803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:21.805803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:21.805803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:21.805803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:21.805803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:21.805803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:21.805803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:21.805803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:21.805803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:21.806669 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:21.806669 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:21.806669 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:21.806669 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:21.807548 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 498/985] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:21.812183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a9227-0b7d-45b7-89e7-22cec489b597 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:21.848631 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:21.919851 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a9227-0b7d-45b7-89e7-22cec489b597 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:21.954336 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7803a-5422-4d6b-8d94-6499a9623747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:21.955025 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7803a-5422-4d6b-8d94-6499a9623747 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:21.955312 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7803a-5422-4d6b-8d94-6499a9623747 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:21.955590 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7803a-5422-4d6b-8d94-6499a9623747 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:21.956310 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba7803a-5422-4d6b-8d94-6499a9623747 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:21.956620 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba7803a-5422-4d6b-8d94-6499a9623747 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:21.958020 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba7803a-5422-4d6b-8d94-6499a9623747 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:21.969678 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 488/986] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:21.974952 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e124a82-365b-41c4-9daf-5488dde5a6e9 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:22.079403 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e124a82-365b-41c4-9daf-5488dde5a6e9 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:22.115674 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca2d66-17a4-4932-9014-93d8c926b7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:22.116537 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca2d66-17a4-4932-9014-93d8c926b7c9 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:22.116823 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca2d66-17a4-4932-9014-93d8c926b7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:22.117102 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca2d66-17a4-4932-9014-93d8c926b7c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:22.123756 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca2d66-17a4-4932-9014-93d8c926b7c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:22.125203 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca2d66-17a4-4932-9014-93d8c926b7c9 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:22.138808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-88ca2d66-17a4-4932-9014-93d8c926b7c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:22.138808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:22.138808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:22.138808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:22.138808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:22.138808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:22.138808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:22.138808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:22.138808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:22.138808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:22.138808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:22.138808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:22.138808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:30:22.138808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:30:22.138808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:30:22.139686 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:30:22.139686 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:22.139686 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:22.139686 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:22.139686 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:22.139686 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:22.139686 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:22.139686 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:22.139686 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:22.139686 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:22.139686 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:22.139686 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:22.139686 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:22.139686 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:22.139686 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:22.139686 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:22.139686 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:22.140562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:30:22.140562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:30:22.140562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:30:22.140562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:30:22.140562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:30:22.140562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:30:22.140562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:22.140562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:22.143087 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 499/987] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:22.145722 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7803a-5422-4d6b-8d94-6499a9623747 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:22.246463 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:22.253489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7803a-5422-4d6b-8d94-6499a9623747 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:22.288032 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e95d1-32cb-46bb-b329-9aa9e7c222b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:22.288726 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e95d1-32cb-46bb-b329-9aa9e7c222b2 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:22.289010 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e95d1-32cb-46bb-b329-9aa9e7c222b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:22.289288 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e95d1-32cb-46bb-b329-9aa9e7c222b2 None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:22.290071 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e95d1-32cb-46bb-b329-9aa9e7c222b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:22.290389 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e95d1-32cb-46bb-b329-9aa9e7c222b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:22.291851 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e95d1-32cb-46bb-b329-9aa9e7c222b2 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:22.307962 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 489/988] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:30:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:22.314422 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca2d66-17a4-4932-9014-93d8c926b7c9 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:22.357384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:22.373488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f1446-6efc-4cf6-9c5b-10bc41c6119a None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:22.373774 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f1446-6efc-4cf6-9c5b-10bc41c6119a None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:22.374052 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f1446-6efc-4cf6-9c5b-10bc41c6119a None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:22.375463 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 490/989] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:30:22.380395 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef77ef-2812-4a18-aa1c-03e698f7282d None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:22.380681 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef77ef-2812-4a18-aa1c-03e698f7282d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:22.380960 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef77ef-2812-4a18-aa1c-03e698f7282d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:22.398342 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-03ef77ef-2812-4a18-aa1c-03e698f7282d None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:22.399550 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-03ef77ef-2812-4a18-aa1c-03e698f7282d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:22.431671 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca2d66-17a4-4932-9014-93d8c926b7c9 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:22.469648 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b68c3-f9c4-4c66-8c27-a5e368465192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:22.470310 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b68c3-f9c4-4c66-8c27-a5e368465192 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:22.470591 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b68c3-f9c4-4c66-8c27-a5e368465192 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:22.470870 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b68c3-f9c4-4c66-8c27-a5e368465192 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:22.471728 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b68c3-f9c4-4c66-8c27-a5e368465192 None admin] RBAC: Adding query filter params (role.id=48dafbec535e4eee8263719557d59d6f, scope.project.id=d488c79cb1704e9496fc196d8aa6c104, user.id=bf393488726e49338e70017f5cb7c4be) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:22.472043 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b68c3-f9c4-4c66-8c27-a5e368465192 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:22.473478 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b68c3-f9c4-4c66-8c27-a5e368465192 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:22.475358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 491/990] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:22 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:30:22.484266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 500/991] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:22 2022] GET /identity/v3/role_assignments?user.id=bf393488726e49338e70017f5cb7c4be&scope.project.id=d488c79cb1704e9496fc196d8aa6c104&role.id=48dafbec535e4eee8263719557d59d6f => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:22.499026 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef77ef-2812-4a18-aa1c-03e698f7282d None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:22.593568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:22.611914 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef77ef-2812-4a18-aa1c-03e698f7282d None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:22.656849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19b9d1-0cb7-4de3-832c-c21c9a5079bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:22.657663 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19b9d1-0cb7-4de3-832c-c21c9a5079bc None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:22.657949 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19b9d1-0cb7-4de3-832c-c21c9a5079bc None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:22.658227 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19b9d1-0cb7-4de3-832c-c21c9a5079bc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:22.658888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19b9d1-0cb7-4de3-832c-c21c9a5079bc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:22.660362 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19b9d1-0cb7-4de3-832c-c21c9a5079bc None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:22.673263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-9a19b9d1-0cb7-4de3-832c-c21c9a5079bc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:30:22.673263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:22.673263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:22.673263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:22.673263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:22.673263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:22.673263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:22.673263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:22.673263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:30:22.673263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:22.673263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:22.673263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:22.673263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:22.673263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:22.673263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:22.673263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:22.673263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:22.674284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:22.674284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:22.674284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:22.674284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:22.674284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:22.674284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:22.674284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:22.674284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:22.674284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:22.674284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:22.674284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:22.674284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:22.674284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:22.674284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:22.674284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:22.674284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:22.675151 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:22.675151 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:22.675151 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:22.675151 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:22.675151 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:22.675151 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:22.675151 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:22.675151 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:22.675151 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:22.675151 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:22.675151 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:22.675151 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:22.675151 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:30:22.675151 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:22.679075 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 492/992] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Jun 15 14:30:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:22.680765 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b68c3-f9c4-4c66-8c27-a5e368465192 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:22.793763 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b68c3-f9c4-4c66-8c27-a5e368465192 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:22.795480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:22.834109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaacd54f-7593-409d-8e96-69d932112516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:22.834808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaacd54f-7593-409d-8e96-69d932112516 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:22.835125 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaacd54f-7593-409d-8e96-69d932112516 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:22.835404 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaacd54f-7593-409d-8e96-69d932112516 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:22.836138 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaacd54f-7593-409d-8e96-69d932112516 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:22.836453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaacd54f-7593-409d-8e96-69d932112516 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:22.838116 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaacd54f-7593-409d-8e96-69d932112516 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:22.849738 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 501/993] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Jun 15 14:30:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:22.851163 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19b9d1-0cb7-4de3-832c-c21c9a5079bc None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:22.899154 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595050f-887d-4731-adfd-d071a625c39e None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:22.899452 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595050f-887d-4731-adfd-d071a625c39e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:22.899769 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595050f-887d-4731-adfd-d071a625c39e None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:22.901188 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 502/994] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:22.906631 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb2f0c-c2a8-47cf-a48b-6bc78c5e97f3 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:22.906919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb2f0c-c2a8-47cf-a48b-6bc78c5e97f3 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:22.907221 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb2f0c-c2a8-47cf-a48b-6bc78c5e97f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:22.925034 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-69eb2f0c-c2a8-47cf-a48b-6bc78c5e97f3 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:22.926231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-69eb2f0c-c2a8-47cf-a48b-6bc78c5e97f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:22.960683 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19b9d1-0cb7-4de3-832c-c21c9a5079bc None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:22.996795 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 503/995] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:22 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:30:23.003191 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a7ca2-080d-4bde-8f08-7d0fbd30f71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:23.003191 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a7ca2-080d-4bde-8f08-7d0fbd30f71e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:23.003191 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a7ca2-080d-4bde-8f08-7d0fbd30f71e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:23.003191 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a7ca2-080d-4bde-8f08-7d0fbd30f71e None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:23.003530 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a7ca2-080d-4bde-8f08-7d0fbd30f71e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:23.003530 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a7ca2-080d-4bde-8f08-7d0fbd30f71e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:23.003530 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a7ca2-080d-4bde-8f08-7d0fbd30f71e None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:23.011033 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 493/996] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Jun 15 14:30:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:23.013812 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb2f0c-c2a8-47cf-a48b-6bc78c5e97f3 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:23.018620 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a7ca2-080d-4bde-8f08-7d0fbd30f71e None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:23.035906 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:23.122595 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:23.129073 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb2f0c-c2a8-47cf-a48b-6bc78c5e97f3 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:23.129193 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a7ca2-080d-4bde-8f08-7d0fbd30f71e None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:23.168482 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499721f-29eb-4d8e-808d-44380d7b19a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:23.169088 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499721f-29eb-4d8e-808d-44380d7b19a8 None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:23.169373 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499721f-29eb-4d8e-808d-44380d7b19a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:23.169658 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499721f-29eb-4d8e-808d-44380d7b19a8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:23.170415 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c499721f-29eb-4d8e-808d-44380d7b19a8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:23.171397 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5135abed-0929-466b-a535-04decec37b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:23.171839 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c499721f-29eb-4d8e-808d-44380d7b19a8 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:23.172773 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5135abed-0929-466b-a535-04decec37b82 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:23.173199 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5135abed-0929-466b-a535-04decec37b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:23.173614 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5135abed-0929-466b-a535-04decec37b82 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:23.174401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5135abed-0929-466b-a535-04decec37b82 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:23.175924 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5135abed-0929-466b-a535-04decec37b82 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:23.187811 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 504/997] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Jun 15 14:30:23 2022] POST /identity/v3/endpoints => generated 325 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:23.188856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-5135abed-0929-466b-a535-04decec37b82 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:30:23.188856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:23.188856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:23.188856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:23.188856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:23.188856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:23.188856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:23.188856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:23.188856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:30:23.188856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:23.188856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:23.188856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:23.188856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:23.188856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:23.188856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:23.188856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:23.188856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:23.190035 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:23.190035 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:23.190035 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:23.190035 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:23.190035 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:23.190035 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:23.190035 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:23.190035 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:23.190035 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:23.190035 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:23.190035 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:23.190035 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:23.190035 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:23.190035 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:23.190035 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:23.190035 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:23.191101 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:23.191101 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:23.191101 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:23.191101 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:23.191101 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:23.191101 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:23.191101 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:23.191101 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:23.191101 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:23.191101 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:23.191101 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:23.191101 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:23.191101 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:30:23.191101 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:23.193388 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 494/998] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:23.200527 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499721f-29eb-4d8e-808d-44380d7b19a8 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:23.325963 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499721f-29eb-4d8e-808d-44380d7b19a8 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:23.366506 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a4b912-761d-4351-a4f9-ec2b2913435b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:23.367729 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a4b912-761d-4351-a4f9-ec2b2913435b None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:23.367729 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a4b912-761d-4351-a4f9-ec2b2913435b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:23.367924 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a4b912-761d-4351-a4f9-ec2b2913435b None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:23.369707 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a4b912-761d-4351-a4f9-ec2b2913435b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:23.369707 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a4b912-761d-4351-a4f9-ec2b2913435b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:23.370193 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a4b912-761d-4351-a4f9-ec2b2913435b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:23.377298 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 505/999] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Jun 15 14:30:23 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:23.382782 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5135abed-0929-466b-a535-04decec37b82 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:23.476088 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:23.498838 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5135abed-0929-466b-a535-04decec37b82 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:23.533524 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca7a2d3-2e0c-4aac-acda-9d426203687a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:23.534198 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7a2d3-2e0c-4aac-acda-9d426203687a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:23.534484 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7a2d3-2e0c-4aac-acda-9d426203687a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:23.534759 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7a2d3-2e0c-4aac-acda-9d426203687a None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:23.535501 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca7a2d3-2e0c-4aac-acda-9d426203687a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:23.535812 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca7a2d3-2e0c-4aac-acda-9d426203687a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:23.537176 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca7a2d3-2e0c-4aac-acda-9d426203687a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:23.547535 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 495/1000] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Jun 15 14:30:23 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:23.548942 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a4b912-761d-4351-a4f9-ec2b2913435b None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:23.568370 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:23.658969 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a4b912-761d-4351-a4f9-ec2b2913435b None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:23.697206 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cfa1eb-ecd9-4ae9-a660-3c3b193d8152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:23.697871 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cfa1eb-ecd9-4ae9-a660-3c3b193d8152 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:23.698151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cfa1eb-ecd9-4ae9-a660-3c3b193d8152 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:23.698422 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cfa1eb-ecd9-4ae9-a660-3c3b193d8152 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:23.699260 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cfa1eb-ecd9-4ae9-a660-3c3b193d8152 None admin] RBAC: Adding query filter params (interface=public, service_id=4aa12bd325c74ec18b2c1b74700819ff, region_id=RegionOne) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:23.699573 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cfa1eb-ecd9-4ae9-a660-3c3b193d8152 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:23.700941 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cfa1eb-ecd9-4ae9-a660-3c3b193d8152 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:23.710978 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 506/1001] 10.222.0.45 () {56 vars in 1251 bytes} [Wed Jun 15 14:30:23 2022] GET /identity/v3/endpoints?service_id=4aa12bd325c74ec18b2c1b74700819ff&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:23.716890 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca7a2d3-2e0c-4aac-acda-9d426203687a None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:23.826758 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca7a2d3-2e0c-4aac-acda-9d426203687a None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:23.861273 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e70c5e-cebf-4f14-a28b-bb60ee9a7d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:23.861937 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e70c5e-cebf-4f14-a28b-bb60ee9a7d24 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:23.862215 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e70c5e-cebf-4f14-a28b-bb60ee9a7d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:23.862489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e70c5e-cebf-4f14-a28b-bb60ee9a7d24 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:23.863194 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e70c5e-cebf-4f14-a28b-bb60ee9a7d24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:23.864550 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e70c5e-cebf-4f14-a28b-bb60ee9a7d24 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:23.879094 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 496/1002] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Jun 15 14:30:23 2022] GET /identity/v3/services => generated 1727 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:30:24.677373 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7dc10-8464-46f4-9001-3f9a81971385 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:24.677883 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7dc10-8464-46f4-9001-3f9a81971385 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:24.677961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7dc10-8464-46f4-9001-3f9a81971385 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:24.679473 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 507/1003] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:24.684402 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6918fe-0a72-46a0-875e-b44fede18a46 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:24.684803 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6918fe-0a72-46a0-875e-b44fede18a46 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:24.684990 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6918fe-0a72-46a0-875e-b44fede18a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:24.696338 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:24.704852 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-5a6918fe-0a72-46a0-875e-b44fede18a46 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:24.704852 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-5a6918fe-0a72-46a0-875e-b44fede18a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:24.783043 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 497/1004] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:24 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:30:24.797681 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc7dc10-8464-46f4-9001-3f9a81971385 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:24.892266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:24.914546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc7dc10-8464-46f4-9001-3f9a81971385 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:24.947426 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7dd613-d42b-483d-a445-097c42e22ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:24.948130 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7dd613-d42b-483d-a445-097c42e22ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:24.948409 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7dd613-d42b-483d-a445-097c42e22ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:24.948681 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7dd613-d42b-483d-a445-097c42e22ca9 None admin] PATH_INFO: `/v3/services/image` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:24.949856 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7dd613-d42b-483d-a445-097c42e22ca9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:24.951249 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7dd613-d42b-483d-a445-097c42e22ca9 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:24.963580 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-fd7dd613-d42b-483d-a445-097c42e22ca9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:30:24.963580 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:24.963580 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:24.963580 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:24.963580 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:24.963580 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:24.963580 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:24.963580 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:24.963580 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:30:24.963580 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:24.963580 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:24.963580 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:24.963580 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:24.963580 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:24.963580 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:24.963580 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:24.963580 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:24.964661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:24.964661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:24.964661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:24.964661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:24.964661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:24.964661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:24.964661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:24.964661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:24.964661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:24.964661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:24.964661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:24.964661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:24.964661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:24.964661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:24.964661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:24.964661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:24.966129 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:24.966129 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:24.966129 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:24.966129 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:24.966129 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:24.966129 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:24.966129 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:24.966129 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:24.966129 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:24.966129 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:24.966129 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:24.966129 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:24.966129 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:30:24.966129 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:24.970697 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 508/1005] 10.222.0.45 () {56 vars in 1105 bytes} [Wed Jun 15 14:30:24 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:24.974726 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6918fe-0a72-46a0-875e-b44fede18a46 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:25.082414 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6918fe-0a72-46a0-875e-b44fede18a46 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:25.116539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465de53d-110f-4b07-978e-f98c96a126fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:25.117212 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465de53d-110f-4b07-978e-f98c96a126fb None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:25.117499 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465de53d-110f-4b07-978e-f98c96a126fb None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:25.117779 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465de53d-110f-4b07-978e-f98c96a126fb None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:25.118481 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465de53d-110f-4b07-978e-f98c96a126fb None admin] RBAC: Adding query filter params (name=image) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:25.118790 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465de53d-110f-4b07-978e-f98c96a126fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:25.120213 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465de53d-110f-4b07-978e-f98c96a126fb None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:25.122681 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:25.132119 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 498/1006] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:24 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:25.133342 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7dd613-d42b-483d-a445-097c42e22ca9 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:25.239368 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7dd613-d42b-483d-a445-097c42e22ca9 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:25.275836 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73727e4-a640-433e-a5ca-a3f76c9e12fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:25.276882 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73727e4-a640-433e-a5ca-a3f76c9e12fc None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:25.277393 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73727e4-a640-433e-a5ca-a3f76c9e12fc None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:25.277853 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73727e4-a640-433e-a5ca-a3f76c9e12fc None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:25.278782 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73727e4-a640-433e-a5ca-a3f76c9e12fc None admin] RBAC: Adding query filter params (type=image) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:25.279314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73727e4-a640-433e-a5ca-a3f76c9e12fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:25.280951 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73727e4-a640-433e-a5ca-a3f76c9e12fc None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:25.287047 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 509/1007] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:25 2022] GET /identity/v3/services?type=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:25.351168 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bea58-d5cf-4893-9952-182a87c86162 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:25.351459 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bea58-d5cf-4893-9952-182a87c86162 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:25.351742 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bea58-d5cf-4893-9952-182a87c86162 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:25.353142 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 499/1008] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:25.358858 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4061e-9b29-4546-a8bd-3ab2a7ad910c None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:25.359181 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4061e-9b29-4546-a8bd-3ab2a7ad910c None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:25.359468 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4061e-9b29-4546-a8bd-3ab2a7ad910c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:25.377819 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-44c4061e-9b29-4546-a8bd-3ab2a7ad910c None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:25.379058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-44c4061e-9b29-4546-a8bd-3ab2a7ad910c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:25.381309 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:25.460466 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 510/1009] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:25 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:25.483964 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741bea58-d5cf-4893-9952-182a87c86162 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:25.596241 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741bea58-d5cf-4893-9952-182a87c86162 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:25.635154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ab940-6f5e-4cbe-bce1-3f1a25b6c6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:25.636040 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ab940-6f5e-4cbe-bce1-3f1a25b6c6a4 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:25.636388 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ab940-6f5e-4cbe-bce1-3f1a25b6c6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:25.637269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ab940-6f5e-4cbe-bce1-3f1a25b6c6a4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:25.638518 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450ab940-6f5e-4cbe-bce1-3f1a25b6c6a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:25.640053 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450ab940-6f5e-4cbe-bce1-3f1a25b6c6a4 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:25.641897 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 500/1010] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:25.647825 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c4061e-9b29-4546-a8bd-3ab2a7ad910c None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:25.756719 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c4061e-9b29-4546-a8bd-3ab2a7ad910c None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:25.793430 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f754f-3013-4b76-8c93-bef7086f7d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:25.794361 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f754f-3013-4b76-8c93-bef7086f7d11 None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:25.794642 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f754f-3013-4b76-8c93-bef7086f7d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:25.794912 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f754f-3013-4b76-8c93-bef7086f7d11 None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:25.795653 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f754f-3013-4b76-8c93-bef7086f7d11 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:25.797077 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f754f-3013-4b76-8c93-bef7086f7d11 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:25.805683 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:25.832758 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 511/1011] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Jun 15 14:30:25 2022] POST /identity/v3/projects => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:26.025644 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd81cf9-2918-47d7-9f09-30bc78b8c04f None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:26.026126 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd81cf9-2918-47d7-9f09-30bc78b8c04f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:26.026126 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd81cf9-2918-47d7-9f09-30bc78b8c04f None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:26.028111 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 501/1012] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:26.033262 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03618f5e-a09d-410a-ad49-c986b1bde125 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:26.033564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03618f5e-a09d-410a-ad49-c986b1bde125 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:26.033852 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03618f5e-a09d-410a-ad49-c986b1bde125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:26.052524 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-03618f5e-a09d-410a-ad49-c986b1bde125 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:26.054123 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-03618f5e-a09d-410a-ad49-c986b1bde125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:26.126144 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 512/1013] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:26 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:26.149560 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd81cf9-2918-47d7-9f09-30bc78b8c04f None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:26.218751 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:26.256214 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd81cf9-2918-47d7-9f09-30bc78b8c04f None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:26.293968 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08448a98-647f-4377-8692-34be184d3553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:26.294769 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08448a98-647f-4377-8692-34be184d3553 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:26.295108 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08448a98-647f-4377-8692-34be184d3553 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:26.295364 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08448a98-647f-4377-8692-34be184d3553 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:26.296021 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08448a98-647f-4377-8692-34be184d3553 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:26.297393 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08448a98-647f-4377-8692-34be184d3553 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:26.309779 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-08448a98-647f-4377-8692-34be184d3553 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:30:26.309779 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:26.309779 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:26.309779 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:26.309779 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:26.309779 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:26.309779 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:26.309779 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:26.309779 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:30:26.309779 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:26.309779 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:26.309779 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:26.309779 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:26.309779 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:26.309779 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:26.309779 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:26.309779 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:26.310888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:26.310888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:26.310888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:26.310888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:26.310888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:26.310888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:26.310888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:26.310888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:26.310888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:26.310888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:26.310888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:26.310888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:26.310888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:26.310888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:26.310888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:26.310888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:26.311745 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:26.311745 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:26.311745 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:26.311745 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:26.311745 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:26.311745 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:26.311745 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:26.311745 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:26.311745 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:26.311745 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:26.311745 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:26.311745 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:26.311745 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:30:26.311745 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:26.315072 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 502/1014] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:26 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:26.317299 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03618f5e-a09d-410a-ad49-c986b1bde125 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:26.424205 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03618f5e-a09d-410a-ad49-c986b1bde125 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:26.444771 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:26.461140 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261cbdc8-9a80-4943-9db9-200e59fec67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:26.461815 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261cbdc8-9a80-4943-9db9-200e59fec67e None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:26.462103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261cbdc8-9a80-4943-9db9-200e59fec67e None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:26.462380 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261cbdc8-9a80-4943-9db9-200e59fec67e None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:26.463121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261cbdc8-9a80-4943-9db9-200e59fec67e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:26.463434 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261cbdc8-9a80-4943-9db9-200e59fec67e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:26.465402 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261cbdc8-9a80-4943-9db9-200e59fec67e None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:26.470576 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 513/1015] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Jun 15 14:30:26 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:26.476922 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08448a98-647f-4377-8692-34be184d3553 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:26.584046 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08448a98-647f-4377-8692-34be184d3553 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:26.614327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:26.620379 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe3fc7-50d1-433d-89e3-e49f77e3f40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:26.621586 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe3fc7-50d1-433d-89e3-e49f77e3f40a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:26.621873 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe3fc7-50d1-433d-89e3-e49f77e3f40a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:26.622151 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe3fc7-50d1-433d-89e3-e49f77e3f40a None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:26.622860 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fe3fc7-50d1-433d-89e3-e49f77e3f40a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:26.623201 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fe3fc7-50d1-433d-89e3-e49f77e3f40a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:26.624582 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fe3fc7-50d1-433d-89e3-e49f77e3f40a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:26.634121 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 503/1016] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Jun 15 14:30:26 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:26.635482 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261cbdc8-9a80-4943-9db9-200e59fec67e None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:26.752620 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261cbdc8-9a80-4943-9db9-200e59fec67e None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:26.787435 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c3d85d-9f70-4b58-90cd-5c1c6d54f0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:26.788171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3d85d-9f70-4b58-90cd-5c1c6d54f0ac None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:26.788371 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3d85d-9f70-4b58-90cd-5c1c6d54f0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:26.788628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3d85d-9f70-4b58-90cd-5c1c6d54f0ac None admin] PATH_INFO: `/v3/endpoints` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:26.789354 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c3d85d-9f70-4b58-90cd-5c1c6d54f0ac None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:26.790718 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c3d85d-9f70-4b58-90cd-5c1c6d54f0ac None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:26.810435 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 514/1017] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Jun 15 14:30:26 2022] POST /identity/v3/endpoints => generated 334 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:27.444747 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1673db2-b0d5-4edc-b76d-f067cd7abad5 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:27.445262 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1673db2-b0d5-4edc-b76d-f067cd7abad5 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:27.445341 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1673db2-b0d5-4edc-b76d-f067cd7abad5 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:27.446847 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 504/1018] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:27.452943 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34cf09-0101-4e42-b8c6-99c975a8bcf0 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:27.453248 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34cf09-0101-4e42-b8c6-99c975a8bcf0 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:27.453532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34cf09-0101-4e42-b8c6-99c975a8bcf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:27.471497 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-2b34cf09-0101-4e42-b8c6-99c975a8bcf0 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:27.472682 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-2b34cf09-0101-4e42-b8c6-99c975a8bcf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:27.508387 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:27.555028 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 515/1019] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:27 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:27.577471 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1673db2-b0d5-4edc-b76d-f067cd7abad5 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:27.685128 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1673db2-b0d5-4edc-b76d-f067cd7abad5 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:27.726145 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c35b3-1529-452d-ba83-71f371a2f928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:27.727596 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c35b3-1529-452d-ba83-71f371a2f928 None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:27.727596 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c35b3-1529-452d-ba83-71f371a2f928 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:27.727768 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c35b3-1529-452d-ba83-71f371a2f928 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:27.728441 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590c35b3-1529-452d-ba83-71f371a2f928 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:27.729809 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590c35b3-1529-452d-ba83-71f371a2f928 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:27.751038 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 505/1020] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Jun 15 14:30:27 2022] POST /identity/v3/services => generated 242 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:28.056666 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91720ea4-3615-466f-8270-97e98cfd7d0b None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:28.057158 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91720ea4-3615-466f-8270-97e98cfd7d0b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:28.057237 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91720ea4-3615-466f-8270-97e98cfd7d0b None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:28.058629 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 516/1021] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:28.064036 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ecb61-2f5c-42bf-b6ef-a28eee8070ce None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:28.064337 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ecb61-2f5c-42bf-b6ef-a28eee8070ce None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:28.064622 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ecb61-2f5c-42bf-b6ef-a28eee8070ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:28.082352 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-032ecb61-2f5c-42bf-b6ef-a28eee8070ce None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:28.083665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-032ecb61-2f5c-42bf-b6ef-a28eee8070ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:28.134895 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:28.160396 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 506/1022] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:28 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:28.185588 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91720ea4-3615-466f-8270-97e98cfd7d0b None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:28.291267 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91720ea4-3615-466f-8270-97e98cfd7d0b None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:28.325617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ebecde-e760-487e-8400-6f9cac65d0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:28.326333 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ebecde-e760-487e-8400-6f9cac65d0ec None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:28.326613 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ebecde-e760-487e-8400-6f9cac65d0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:28.326887 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ebecde-e760-487e-8400-6f9cac65d0ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:28.328087 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ebecde-e760-487e-8400-6f9cac65d0ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:28.329521 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ebecde-e760-487e-8400-6f9cac65d0ec None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:28.335120 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 517/1023] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:28.337095 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ecb61-2f5c-42bf-b6ef-a28eee8070ce None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:28.440351 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ecb61-2f5c-42bf-b6ef-a28eee8070ce None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:28.474143 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc5dff6-2af6-4cf9-a944-0420ad0524ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:28.474923 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5dff6-2af6-4cf9-a944-0420ad0524ea None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:28.475139 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5dff6-2af6-4cf9-a944-0420ad0524ea None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:28.475395 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5dff6-2af6-4cf9-a944-0420ad0524ea None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:28.476107 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc5dff6-2af6-4cf9-a944-0420ad0524ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:28.477532 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc5dff6-2af6-4cf9-a944-0420ad0524ea None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:28.503199 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 507/1024] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Jun 15 14:30:28 2022] POST /identity/v3/users => generated 285 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:29.044197 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d95b7-da80-404d-9b25-18e3decf64e3 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:29.044553 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d95b7-da80-404d-9b25-18e3decf64e3 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:29.044762 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d95b7-da80-404d-9b25-18e3decf64e3 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:29.046160 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 518/1025] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:29.051529 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f12039-4e1f-42d6-8b89-235375d1cb49 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:29.051818 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f12039-4e1f-42d6-8b89-235375d1cb49 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:29.052091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f12039-4e1f-42d6-8b89-235375d1cb49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:29.070179 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-01f12039-4e1f-42d6-8b89-235375d1cb49 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:29.071401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-01f12039-4e1f-42d6-8b89-235375d1cb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:29.101232 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:29.141892 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 508/1026] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:30:29.164768 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d95b7-da80-404d-9b25-18e3decf64e3 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:29.200747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:29.273386 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d95b7-da80-404d-9b25-18e3decf64e3 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:29.309760 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64baed3-07fc-47df-94df-37f1ccf50142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:29.310812 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64baed3-07fc-47df-94df-37f1ccf50142 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:29.311292 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64baed3-07fc-47df-94df-37f1ccf50142 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:29.311790 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64baed3-07fc-47df-94df-37f1ccf50142 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:29.322579 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64baed3-07fc-47df-94df-37f1ccf50142 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:29.331011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-a64baed3-07fc-47df-94df-37f1ccf50142 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:30:29.331011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:29.331011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:29.331011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:29.331011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:29.331011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:29.331011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:29.331011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:29.331011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:29.331011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:29.331011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:29.331011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:29.331011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:29.331011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:29.331011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:29.331011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:29.332045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:29.332045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:29.332045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:29.332045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:29.332045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:29.332045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:29.332045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:29.332045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:29.332045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:29.332045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:29.332045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:29.332045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:29.332045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:29.332045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:29.332045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:29.332045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:29.332892 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:29.332892 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:29.332892 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:29.332892 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:29.332892 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:29.332892 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:29.332892 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:29.332892 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:29.332892 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:29.332892 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:29.332892 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:30:29.332892 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:29.337749 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 519/1027] 10.222.0.45 () {56 vars in 1120 bytes} [Wed Jun 15 14:30:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:29.338968 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f12039-4e1f-42d6-8b89-235375d1cb49 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:29.451335 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f12039-4e1f-42d6-8b89-235375d1cb49 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:29.491455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f1513-f2e3-4bc6-b319-1e209ad42c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:29.492205 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f1513-f2e3-4bc6-b319-1e209ad42c2a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:29.492413 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f1513-f2e3-4bc6-b319-1e209ad42c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:29.492690 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f1513-f2e3-4bc6-b319-1e209ad42c2a None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:29.493440 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80f1513-f2e3-4bc6-b319-1e209ad42c2a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:29.493751 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80f1513-f2e3-4bc6-b319-1e209ad42c2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:29.495150 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80f1513-f2e3-4bc6-b319-1e209ad42c2a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:29.504202 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 509/1028] 10.222.0.45 () {56 vars in 1115 bytes} [Wed Jun 15 14:30:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:29.509533 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64baed3-07fc-47df-94df-37f1ccf50142 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:29.610810 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:29.618355 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64baed3-07fc-47df-94df-37f1ccf50142 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:29.651869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c929c4-9a94-443e-b526-39a211151186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:29.653233 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c929c4-9a94-443e-b526-39a211151186 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:29.653512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c929c4-9a94-443e-b526-39a211151186 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:29.653809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c929c4-9a94-443e-b526-39a211151186 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:29.665624 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c929c4-9a94-443e-b526-39a211151186 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:29.667113 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c929c4-9a94-443e-b526-39a211151186 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:29.687324 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-e0c929c4-9a94-443e-b526-39a211151186 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:29.687324 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:29.687324 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:29.687324 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:29.687324 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:29.687324 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:29.687324 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:29.687324 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:29.687324 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:29.687324 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:29.687324 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:29.687324 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:29.687324 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:29.687324 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:29.687324 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:29.688268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:29.688268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:29.688268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:29.688268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:29.688268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:29.688268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:29.688268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:29.688268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:29.688268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:29.688268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:29.688268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:29.688268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:29.688268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:29.688268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:29.688268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:29.688268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:29.688268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:29.689143 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:29.689143 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:29.689143 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:29.689143 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:29.689143 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 520/1029] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:29.694704 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f1513-f2e3-4bc6-b319-1e209ad42c2a None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:29.734549 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:29.809802 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f1513-f2e3-4bc6-b319-1e209ad42c2a None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:29.848375 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc8e91-9b42-453d-adf7-f63048392eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:29.849449 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc8e91-9b42-453d-adf7-f63048392eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:29.849907 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc8e91-9b42-453d-adf7-f63048392eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:29.850389 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc8e91-9b42-453d-adf7-f63048392eb1 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:29.851484 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc8e91-9b42-453d-adf7-f63048392eb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:29.851999 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc8e91-9b42-453d-adf7-f63048392eb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:29.853904 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc8e91-9b42-453d-adf7-f63048392eb1 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:29.874849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 510/1030] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:29.877691 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c929c4-9a94-443e-b526-39a211151186 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:29.916831 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9273266-79c8-4786-9cf3-c52b874a7660 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:29.917183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9273266-79c8-4786-9cf3-c52b874a7660 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:29.917388 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9273266-79c8-4786-9cf3-c52b874a7660 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:29.919809 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 511/1031] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:29.924243 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e2abe-5a27-4cf0-8436-4465eaec7b4f None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:29.924534 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e2abe-5a27-4cf0-8436-4465eaec7b4f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:29.924815 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e2abe-5a27-4cf0-8436-4465eaec7b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:29.942927 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-0f7e2abe-5a27-4cf0-8436-4465eaec7b4f None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:29.944152 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-0f7e2abe-5a27-4cf0-8436-4465eaec7b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:30.015076 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c929c4-9a94-443e-b526-39a211151186 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:30.050077 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 512/1032] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:30:30.056206 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12831e86-db30-4ade-a74b-f63f2d3655d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:30.057121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12831e86-db30-4ade-a74b-f63f2d3655d1 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:30.057506 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12831e86-db30-4ade-a74b-f63f2d3655d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:30.057886 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12831e86-db30-4ade-a74b-f63f2d3655d1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:30.065097 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e2abe-5a27-4cf0-8436-4465eaec7b4f None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:30.068870 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12831e86-db30-4ade-a74b-f63f2d3655d1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:30.070356 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12831e86-db30-4ade-a74b-f63f2d3655d1 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:30.086963 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-12831e86-db30-4ade-a74b-f63f2d3655d1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:30:30.086963 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:30.086963 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:30.086963 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:30.086963 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:30.086963 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:30.086963 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:30.086963 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:30.086963 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:30.086963 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:30.086963 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:30.086963 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:30.086963 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:30:30.086963 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:30:30.086963 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:30:30.086963 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:30:30.087977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:30.087977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:30.087977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:30.087977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:30.087977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:30.087977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:30.087977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:30.087977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:30.087977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:30.087977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:30.087977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:30.087977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:30.087977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:30.087977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:30.087977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:30.087977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:30.087977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:30.087977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:30.089013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:30.089013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:30.089013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:30:30.089013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:30:30.089013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:30:30.089013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:30:30.089013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:30:30.089013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:30:30.089013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:30:30.089013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:30.094849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:30.095056 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 521/1033] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Jun 15 14:30:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:30.095568 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12831e86-db30-4ade-a74b-f63f2d3655d1 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:30.107676 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:30.190792 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e2abe-5a27-4cf0-8436-4465eaec7b4f None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:30.219850 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12831e86-db30-4ade-a74b-f63f2d3655d1 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:30.227437 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1782a-9fbb-4460-9626-640e58266e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:30.228617 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1782a-9fbb-4460-9626-640e58266e1b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:30.229051 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1782a-9fbb-4460-9626-640e58266e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:30.229458 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1782a-9fbb-4460-9626-640e58266e1b None admin] PATH_INFO: `/v3/services/image` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:30.230257 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1782a-9fbb-4460-9626-640e58266e1b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:30.232214 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1782a-9fbb-4460-9626-640e58266e1b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:30.244294 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-e0c1782a-9fbb-4460-9626-640e58266e1b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:30:30.244294 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:30.244294 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:30.244294 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:30.244294 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:30.244294 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:30.244294 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:30.244294 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:30.244294 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:30:30.244294 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:30.244294 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:30.244294 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:30.244294 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:30.244294 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:30.244294 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:30.244294 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:30.244294 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:30.245311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:30.245311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:30.245311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:30.245311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:30.245311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:30.245311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:30.245311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:30.245311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:30.245311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:30.245311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:30.245311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:30.245311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:30.245311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:30.245311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:30.245311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:30.245311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:30.246723 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:30.246723 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:30.246723 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:30.246723 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:30.246723 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:30.246723 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:30.246723 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:30.246723 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:30.246723 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:30.246723 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:30.246723 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:30.246723 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:30.246723 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:30:30.246723 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:30.246723 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 513/1034] 10.222.0.45 () {56 vars in 1105 bytes} [Wed Jun 15 14:30:30 2022] GET /identity/v3/services/image => generated 86 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:30.253385 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1782a-9fbb-4460-9626-640e58266e1b None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:30.271673 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30660f-3ff5-4328-8b76-72a30b90d51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:30.272357 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30660f-3ff5-4328-8b76-72a30b90d51f None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:30.272644 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30660f-3ff5-4328-8b76-72a30b90d51f None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:30.272918 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30660f-3ff5-4328-8b76-72a30b90d51f None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:30.274026 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e30660f-3ff5-4328-8b76-72a30b90d51f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:30.274536 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e30660f-3ff5-4328-8b76-72a30b90d51f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:30.276178 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e30660f-3ff5-4328-8b76-72a30b90d51f None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:30.298346 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 522/1035] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Jun 15 14:30:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:30.304130 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30660f-3ff5-4328-8b76-72a30b90d51f None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:30.361928 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:30.377727 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1782a-9fbb-4460-9626-640e58266e1b None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:30.386784 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:30.431331 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa106d-d161-4550-ba3f-71abcd610067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:30.432318 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa106d-d161-4550-ba3f-71abcd610067 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:30.432606 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa106d-d161-4550-ba3f-71abcd610067 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:30.433054 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa106d-d161-4550-ba3f-71abcd610067 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:30.433804 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fa106d-d161-4550-ba3f-71abcd610067 None admin] RBAC: Adding query filter params (name=image) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:30.434170 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fa106d-d161-4550-ba3f-71abcd610067 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:30.435940 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fa106d-d161-4550-ba3f-71abcd610067 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:30.436485 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30660f-3ff5-4328-8b76-72a30b90d51f None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:30.443136 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 514/1036] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:30.449690 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa106d-d161-4550-ba3f-71abcd610067 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:30.489887 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a57ce-aa7e-44d6-9d67-a390b91c32b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:30.490545 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a57ce-aa7e-44d6-9d67-a390b91c32b5 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:30.490834 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a57ce-aa7e-44d6-9d67-a390b91c32b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:30.491153 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a57ce-aa7e-44d6-9d67-a390b91c32b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:30.588752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a57ce-aa7e-44d6-9d67-a390b91c32b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:30.591012 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a57ce-aa7e-44d6-9d67-a390b91c32b5 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:30.598420 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa106d-d161-4550-ba3f-71abcd610067 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:30.645314 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02ea1b-635a-44c1-91aa-462f7fe4cbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:30.647199 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02ea1b-635a-44c1-91aa-462f7fe4cbf3 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:30.650302 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02ea1b-635a-44c1-91aa-462f7fe4cbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:30.650302 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02ea1b-635a-44c1-91aa-462f7fe4cbf3 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:30.650302 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c02ea1b-635a-44c1-91aa-462f7fe4cbf3 None admin] RBAC: Adding query filter params (type=image) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:30.650302 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c02ea1b-635a-44c1-91aa-462f7fe4cbf3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:30.650756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c02ea1b-635a-44c1-91aa-462f7fe4cbf3 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:30.661798 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 515/1037] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:30 2022] GET /identity/v3/services?type=image => generated 350 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:30.663471 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02ea1b-635a-44c1-91aa-462f7fe4cbf3 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:30.681054 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:30.683854 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:30.729775 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 523/1038] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:30 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:30:30.731378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e48076-bd36-4c71-9f24-59ea13405507 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:30.731829 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e48076-bd36-4c71-9f24-59ea13405507 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:30.732117 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e48076-bd36-4c71-9f24-59ea13405507 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:30.733723 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 524/1039] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:30.735599 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e48076-bd36-4c71-9f24-59ea13405507 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:30.779135 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02ea1b-635a-44c1-91aa-462f7fe4cbf3 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:30.818413 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e88c1c0-6279-437d-aa0e-c4f2922b1c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:30.819112 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88c1c0-6279-437d-aa0e-c4f2922b1c45 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:30.819401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88c1c0-6279-437d-aa0e-c4f2922b1c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:30.819678 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88c1c0-6279-437d-aa0e-c4f2922b1c45 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:30.820483 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e88c1c0-6279-437d-aa0e-c4f2922b1c45 None admin] RBAC: Adding query filter params (interface=public, service_id=d0b4911ea7394574a5da919135a5ba22, region_id=RegionOne) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:30.820797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e88c1c0-6279-437d-aa0e-c4f2922b1c45 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:30.823942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e88c1c0-6279-437d-aa0e-c4f2922b1c45 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:30.829514 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 516/1040] 10.222.0.45 () {56 vars in 1251 bytes} [Wed Jun 15 14:30:30 2022] GET /identity/v3/endpoints?service_id=d0b4911ea7394574a5da919135a5ba22&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:30.835601 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e26852-2e7c-4aad-b12c-ea2948d4a893 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:30.835894 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e26852-2e7c-4aad-b12c-ea2948d4a893 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:30.836471 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e26852-2e7c-4aad-b12c-ea2948d4a893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:30.851532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e48076-bd36-4c71-9f24-59ea13405507 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:30.854786 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-a5e26852-2e7c-4aad-b12c-ea2948d4a893 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:30.856628 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-a5e26852-2e7c-4aad-b12c-ea2948d4a893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:30.891424 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce065d6f-09db-4826-814e-6a44ba7358b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:30.892307 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce065d6f-09db-4826-814e-6a44ba7358b8 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:30.892502 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce065d6f-09db-4826-814e-6a44ba7358b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:30.892757 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce065d6f-09db-4826-814e-6a44ba7358b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:30.904633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce065d6f-09db-4826-814e-6a44ba7358b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:30.906105 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce065d6f-09db-4826-814e-6a44ba7358b8 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:30.925727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-ce065d6f-09db-4826-814e-6a44ba7358b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:30.925727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:30.925727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:30.925727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:30.925727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:30.925727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:30.925727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:30.925727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:30.925727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:30.925727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:30.925727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:30.925727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:30.925727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:30.925727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:30.925727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:30.926869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:30.926869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:30.926869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:30.926869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:30.926869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:30.926869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:30.926869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:30.926869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:30.926869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:30.926869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:30.926869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:30.926869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:30.926869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:30.926869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:30.926869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:30.926869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:30.926869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:30.927991 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:30.927991 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:30.927991 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:30.927991 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:30.927991 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 525/1041] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:30.928345 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce065d6f-09db-4826-814e-6a44ba7358b8 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:30.941666 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 517/1042] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:30 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:30.942832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e26852-2e7c-4aad-b12c-ea2948d4a893 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:30.950858 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:30.960947 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:31.046550 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce065d6f-09db-4826-814e-6a44ba7358b8 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:31.062768 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e26852-2e7c-4aad-b12c-ea2948d4a893 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:31.082735 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df86dc6-de36-4f33-a221-444277973b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:31.083505 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df86dc6-de36-4f33-a221-444277973b29 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:31.083741 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df86dc6-de36-4f33-a221-444277973b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:31.084019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df86dc6-de36-4f33-a221-444277973b29 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:31.084711 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df86dc6-de36-4f33-a221-444277973b29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:31.086088 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df86dc6-de36-4f33-a221-444277973b29 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:31.093874 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 526/1043] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Jun 15 14:30:30 2022] GET /identity/v3/services => generated 1957 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:30:31.094754 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df86dc6-de36-4f33-a221-444277973b29 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:31.102420 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450703c5-9d46-45c0-b58f-1279abae5135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:31.103144 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450703c5-9d46-45c0-b58f-1279abae5135 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:31.103431 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450703c5-9d46-45c0-b58f-1279abae5135 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:31.103716 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450703c5-9d46-45c0-b58f-1279abae5135 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:31.104456 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450703c5-9d46-45c0-b58f-1279abae5135 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:31.104764 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450703c5-9d46-45c0-b58f-1279abae5135 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:31.106755 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450703c5-9d46-45c0-b58f-1279abae5135 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:31.127156 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 518/1044] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:31.128412 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450703c5-9d46-45c0-b58f-1279abae5135 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:31.214469 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:31.220853 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:31.245332 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df86dc6-de36-4f33-a221-444277973b29 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:31.264310 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450703c5-9d46-45c0-b58f-1279abae5135 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:31.296877 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601017f2-6257-4731-88bf-4f9afd0af874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:31.297554 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601017f2-6257-4731-88bf-4f9afd0af874 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:31.297833 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601017f2-6257-4731-88bf-4f9afd0af874 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:31.298111 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601017f2-6257-4731-88bf-4f9afd0af874 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:31.303345 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ad9ef-a2f7-4549-a580-8e5d7c7e256b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:31.304391 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ad9ef-a2f7-4549-a580-8e5d7c7e256b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:31.305027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ad9ef-a2f7-4549-a580-8e5d7c7e256b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:31.305488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ad9ef-a2f7-4549-a580-8e5d7c7e256b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:31.310930 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601017f2-6257-4731-88bf-4f9afd0af874 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:31.313894 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ad9ef-a2f7-4549-a580-8e5d7c7e256b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:31.315584 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ad9ef-a2f7-4549-a580-8e5d7c7e256b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:31.319810 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-601017f2-6257-4731-88bf-4f9afd0af874 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:31.319810 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:31.319810 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:31.319810 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:31.319810 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:31.319810 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:31.319810 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:31.319810 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:31.319810 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:31.319810 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:31.319810 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:31.319810 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:31.319810 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:31.319810 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:31.319810 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:31.319810 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:31.321039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:31.321039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:31.321039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:31.321039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:31.321039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:31.321039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:31.321039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:31.321039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:31.321039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:31.321039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:31.321039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:31.321039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:31.321039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:31.321039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:31.321039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:31.321039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:31.321977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:31.321977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:31.321977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:31.321977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:31.321977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:31.321977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:31.321977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:31.321977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:31.321977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:31.321977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:31.321977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:31.321977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:31.321977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 527/1045] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:30:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:31.327836 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601017f2-6257-4731-88bf-4f9afd0af874 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:31.336694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-159ad9ef-a2f7-4549-a580-8e5d7c7e256b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:31.336694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:31.336694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:31.336694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:31.336694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:31.336694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:31.336694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:31.336694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:31.336694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:31.336694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:31.336694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:31.336694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:31.336694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:30:31.336694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:30:31.336694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:30:31.337665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:30:31.337665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:31.337665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:31.337665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:31.337665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:31.337665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:31.337665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:31.337665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:31.337665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:31.337665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:31.337665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:31.337665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:31.337665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:31.337665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:31.337665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:31.337665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:31.337665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:31.338523 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:30:31.338523 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:30:31.338523 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:30:31.338523 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:30:31.338523 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:30:31.338523 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:30:31.338523 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:31.338523 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:31.341052 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 519/1046] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:31.355020 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ad9ef-a2f7-4549-a580-8e5d7c7e256b None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:31.463907 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601017f2-6257-4731-88bf-4f9afd0af874 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:31.471741 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ad9ef-a2f7-4549-a580-8e5d7c7e256b None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:31.505006 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:31.511859 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962ae5a-c463-42ea-9d8e-e3e0108b8521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:31.512948 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962ae5a-c463-42ea-9d8e-e3e0108b8521 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:31.513238 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962ae5a-c463-42ea-9d8e-e3e0108b8521 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:31.513521 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962ae5a-c463-42ea-9d8e-e3e0108b8521 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:31.514289 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b962ae5a-c463-42ea-9d8e-e3e0108b8521 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:31.514606 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b962ae5a-c463-42ea-9d8e-e3e0108b8521 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:31.517841 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b962ae5a-c463-42ea-9d8e-e3e0108b8521 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:31.518047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc8ff5c-4bdf-4af4-bec1-dcb4ac55550f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:31.518719 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8ff5c-4bdf-4af4-bec1-dcb4ac55550f None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:31.519134 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8ff5c-4bdf-4af4-bec1-dcb4ac55550f None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:31.519624 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8ff5c-4bdf-4af4-bec1-dcb4ac55550f None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:31.520121 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc8ff5c-4bdf-4af4-bec1-dcb4ac55550f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:31.520871 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc8ff5c-4bdf-4af4-bec1-dcb4ac55550f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:31.522149 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc8ff5c-4bdf-4af4-bec1-dcb4ac55550f None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:31.528846 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 528/1047] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:31.534772 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962ae5a-c463-42ea-9d8e-e3e0108b8521 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:31.537770 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:31.538838 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 520/1048] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:30:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:31.546445 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc8ff5c-4bdf-4af4-bec1-dcb4ac55550f None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:31.670619 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962ae5a-c463-42ea-9d8e-e3e0108b8521 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:31.675480 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc8ff5c-4bdf-4af4-bec1-dcb4ac55550f None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:31.720570 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd3bb0-4f4a-4646-a87f-27e2d22e90db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:31.721980 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd3bb0-4f4a-4646-a87f-27e2d22e90db None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:31.722913 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd3bb0-4f4a-4646-a87f-27e2d22e90db None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:31.723428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd3bb0-4f4a-4646-a87f-27e2d22e90db None admin] PATH_INFO: `/v3/users/614f8cdc30784b39b1e8370fa40186a6` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:31.725877 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283d538-a532-4e99-a536-9c2ad02e9ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:31.726552 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283d538-a532-4e99-a536-9c2ad02e9ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:31.726835 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283d538-a532-4e99-a536-9c2ad02e9ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:31.727194 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283d538-a532-4e99-a536-9c2ad02e9ed4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:31.729043 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e283d538-a532-4e99-a536-9c2ad02e9ed4 None admin] RBAC: Adding query filter params (role.id=48dafbec535e4eee8263719557d59d6f, scope.project.id=d488c79cb1704e9496fc196d8aa6c104, user.id=f3185bd2458c4e22830975700888e9be) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:31.729043 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e283d538-a532-4e99-a536-9c2ad02e9ed4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:31.731241 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e283d538-a532-4e99-a536-9c2ad02e9ed4 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:31.743243 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 521/1049] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:31 2022] GET /identity/v3/role_assignments?user.id=f3185bd2458c4e22830975700888e9be&scope.project.id=d488c79cb1704e9496fc196d8aa6c104&role.id=48dafbec535e4eee8263719557d59d6f => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:31.754087 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd3bb0-4f4a-4646-a87f-27e2d22e90db None admin] RBAC: Authorizing `identity:get_user(user_id=614f8cdc30784b39b1e8370fa40186a6)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:31.755767 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd3bb0-4f4a-4646-a87f-27e2d22e90db None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:31.762557 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 529/1050] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:31 2022] GET /identity/v3/users/614f8cdc30784b39b1e8370fa40186a6 => generated 285 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:31.764525 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283d538-a532-4e99-a536-9c2ad02e9ed4 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:31.823582 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:31.890964 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283d538-a532-4e99-a536-9c2ad02e9ed4 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:31.933683 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f722b8-1676-4726-a9fa-ff2061d1b81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:31.934269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f722b8-1676-4726-a9fa-ff2061d1b81e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:31.934555 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f722b8-1676-4726-a9fa-ff2061d1b81e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:31.934833 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f722b8-1676-4726-a9fa-ff2061d1b81e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:32.008639 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f722b8-1676-4726-a9fa-ff2061d1b81e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:32.010104 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f722b8-1676-4726-a9fa-ff2061d1b81e None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:32.063103 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:32.119157 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 522/1051] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:31 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:30:32.125654 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd3bb0-4f4a-4646-a87f-27e2d22e90db None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:32.189349 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:32.236690 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd3bb0-4f4a-4646-a87f-27e2d22e90db None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:32.278304 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186cfda-d85f-4352-a7d0-e9570c33abb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:32.279198 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186cfda-d85f-4352-a7d0-e9570c33abb5 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:32.279480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186cfda-d85f-4352-a7d0-e9570c33abb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:32.279749 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186cfda-d85f-4352-a7d0-e9570c33abb5 None admin] PATH_INFO: `/v3/projects/266b416e205a46aa8c934fbcae0f571d` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:32.280939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5186cfda-d85f-4352-a7d0-e9570c33abb5 None admin] RBAC: Authorizing `identity:get_project(project_id=266b416e205a46aa8c934fbcae0f571d)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:32.282375 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5186cfda-d85f-4352-a7d0-e9570c33abb5 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:32.284432 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 530/1052] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:30:32 2022] GET /identity/v3/projects/266b416e205a46aa8c934fbcae0f571d => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:32.293765 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f722b8-1676-4726-a9fa-ff2061d1b81e None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:32.406201 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f722b8-1676-4726-a9fa-ff2061d1b81e None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:32.443337 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4457ab-c030-4344-a7b3-8f6bdc6992a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:32.444325 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4457ab-c030-4344-a7b3-8f6bdc6992a5 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:32.444803 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4457ab-c030-4344-a7b3-8f6bdc6992a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:32.445505 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4457ab-c030-4344-a7b3-8f6bdc6992a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:32.446547 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4457ab-c030-4344-a7b3-8f6bdc6992a5 None admin] RBAC: Adding query filter params (role.id=ed1cfa2fcf724a4d84f5acea100930c7, scope.project.id=266b416e205a46aa8c934fbcae0f571d, user.id=614f8cdc30784b39b1e8370fa40186a6) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:32.447092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4457ab-c030-4344-a7b3-8f6bdc6992a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:32.448715 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4457ab-c030-4344-a7b3-8f6bdc6992a5 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:32.457219 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 523/1053] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:32 2022] GET /identity/v3/role_assignments?user.id=614f8cdc30784b39b1e8370fa40186a6&scope.project.id=266b416e205a46aa8c934fbcae0f571d&role.id=ed1cfa2fcf724a4d84f5acea100930c7 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:33.292623 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140cb515-c832-4d34-b415-93eefae1f613 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:33.293705 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140cb515-c832-4d34-b415-93eefae1f613 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:33.293785 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140cb515-c832-4d34-b415-93eefae1f613 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:33.296319 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 531/1054] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:33.303146 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4593a-957c-407f-8015-1c5f57b1f17a None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:33.303146 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4593a-957c-407f-8015-1c5f57b1f17a None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:33.303146 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4593a-957c-407f-8015-1c5f57b1f17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:33.321438 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-4ef4593a-957c-407f-8015-1c5f57b1f17a None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:33.322837 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-4ef4593a-957c-407f-8015-1c5f57b1f17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:33.334260 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:33.397316 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 524/1055] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:33 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:33.421144 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140cb515-c832-4d34-b415-93eefae1f613 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:33.532882 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140cb515-c832-4d34-b415-93eefae1f613 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:33.561883 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:33.572031 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6eca3e-2024-430d-b519-df9785bdef53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:33.573238 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6eca3e-2024-430d-b519-df9785bdef53 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:33.573754 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6eca3e-2024-430d-b519-df9785bdef53 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:33.574864 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6eca3e-2024-430d-b519-df9785bdef53 None admin] PATH_INFO: `/v3/services/image` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:33.575802 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6eca3e-2024-430d-b519-df9785bdef53 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:33.577452 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6eca3e-2024-430d-b519-df9785bdef53 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:33.590567 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-1b6eca3e-2024-430d-b519-df9785bdef53 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:30:33.590567 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:33.590567 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:33.590567 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:33.590567 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:33.590567 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:33.590567 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:33.590567 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:33.590567 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:30:33.590567 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:33.590567 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:33.590567 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:33.590567 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:33.590567 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:33.590567 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:33.590567 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:33.590567 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:33.591624 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:33.591624 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:33.591624 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:33.591624 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:33.591624 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:33.591624 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:33.591624 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:33.591624 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:33.591624 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:33.591624 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:33.591624 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:33.591624 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:33.591624 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:33.591624 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:33.591624 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:33.591624 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:33.592450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:33.592450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:33.592450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:33.592450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:33.592450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:33.592450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:33.592450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:33.592450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:33.592450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:33.592450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:33.592450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:33.592450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:33.592450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:30:33.592450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:33.600990 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef4593a-957c-407f-8015-1c5f57b1f17a None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:33.611515 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 532/1056] 10.222.0.45 () {56 vars in 1105 bytes} [Wed Jun 15 14:30:33 2022] GET /identity/v3/services/image => generated 86 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:33.712290 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef4593a-957c-407f-8015-1c5f57b1f17a None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:33.756195 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c909fa-e247-4c53-98f0-c89ae63a055f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:33.757229 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c909fa-e247-4c53-98f0-c89ae63a055f None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:33.757707 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c909fa-e247-4c53-98f0-c89ae63a055f None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:33.758176 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c909fa-e247-4c53-98f0-c89ae63a055f None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:33.759124 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c909fa-e247-4c53-98f0-c89ae63a055f None admin] RBAC: Adding query filter params (name=image) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:33.759626 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c909fa-e247-4c53-98f0-c89ae63a055f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:33.761185 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c909fa-e247-4c53-98f0-c89ae63a055f None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:33.772831 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6eca3e-2024-430d-b519-df9785bdef53 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:33.773523 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 525/1057] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:33 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:33.885195 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6eca3e-2024-430d-b519-df9785bdef53 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:33.923214 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97074308-949d-4631-bf90-588d166dc5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:33.924231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97074308-949d-4631-bf90-588d166dc5c8 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:33.924726 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97074308-949d-4631-bf90-588d166dc5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:33.925193 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97074308-949d-4631-bf90-588d166dc5c8 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:33.926110 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97074308-949d-4631-bf90-588d166dc5c8 None admin] RBAC: Adding query filter params (type=image) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:33.926609 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97074308-949d-4631-bf90-588d166dc5c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:33.928201 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97074308-949d-4631-bf90-588d166dc5c8 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:33.930164 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b40a7-5829-4105-bb07-b0c41cfb4ec7 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:33.930453 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b40a7-5829-4105-bb07-b0c41cfb4ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:33.930731 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b40a7-5829-4105-bb07-b0c41cfb4ec7 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:33.932394 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 526/1058] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:30:33.938410 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34c4e2-1b7a-4342-b4cd-64f8fbbc946d None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:33.938698 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34c4e2-1b7a-4342-b4cd-64f8fbbc946d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:33.939156 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34c4e2-1b7a-4342-b4cd-64f8fbbc946d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:33.939284 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 533/1059] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:33 2022] GET /identity/v3/services?type=image => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:33.940118 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97074308-949d-4631-bf90-588d166dc5c8 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:33.957164 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-de34c4e2-1b7a-4342-b4cd-64f8fbbc946d None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:33.958338 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-de34c4e2-1b7a-4342-b4cd-64f8fbbc946d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:33.960527 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:34.011115 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:34.057882 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 527/1060] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:33 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:30:34.073164 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97074308-949d-4631-bf90-588d166dc5c8 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:34.082343 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34c4e2-1b7a-4342-b4cd-64f8fbbc946d None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:34.114458 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9a872f-cced-4f61-be00-3cb6a756586b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:34.115203 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a872f-cced-4f61-be00-3cb6a756586b None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:34.115493 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a872f-cced-4f61-be00-3cb6a756586b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:34.115767 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a872f-cced-4f61-be00-3cb6a756586b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:34.116510 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9a872f-cced-4f61-be00-3cb6a756586b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:34.117884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9a872f-cced-4f61-be00-3cb6a756586b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:34.142843 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 534/1061] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Jun 15 14:30:33 2022] POST /identity/v3/endpoints => generated 327 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:34.224924 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34c4e2-1b7a-4342-b4cd-64f8fbbc946d None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:34.275278 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5c638-63d0-4d64-94ab-fb1504209a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:34.276114 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5c638-63d0-4d64-94ab-fb1504209a02 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:34.276432 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5c638-63d0-4d64-94ab-fb1504209a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:34.276706 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5c638-63d0-4d64-94ab-fb1504209a02 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:34.279371 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:34.289876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb5c638-63d0-4d64-94ab-fb1504209a02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:34.298263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-4bb5c638-63d0-4d64-94ab-fb1504209a02 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:30:34.298263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:34.298263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:34.298263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:34.298263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:34.298263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:34.298263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:34.298263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:34.298263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:34.298263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:34.298263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:34.298263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:34.298263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:34.298263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:34.298263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:34.298263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:34.299689 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:34.299689 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:34.299689 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:34.299689 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:34.299689 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:34.299689 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:34.299689 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:34.299689 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:34.299689 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:34.299689 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:34.299689 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:34.299689 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:34.299689 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:34.299689 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:34.299689 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:34.299689 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:34.300525 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:34.300525 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:34.300525 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:34.300525 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:34.300525 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:34.300525 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:34.300525 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:34.300525 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:34.300525 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:34.300525 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:34.300525 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:30:34.300525 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:34.304740 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 528/1062] 10.222.0.45 () {56 vars in 1120 bytes} [Wed Jun 15 14:30:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:34.306806 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9a872f-cced-4f61-be00-3cb6a756586b None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:34.441173 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9a872f-cced-4f61-be00-3cb6a756586b None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:34.455824 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:34.480267 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de6eec5-90d6-4ee7-b660-e8a35dea2d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:34.481481 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6eec5-90d6-4ee7-b660-e8a35dea2d27 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:34.481761 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6eec5-90d6-4ee7-b660-e8a35dea2d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:34.482032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6eec5-90d6-4ee7-b660-e8a35dea2d27 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:34.482782 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de6eec5-90d6-4ee7-b660-e8a35dea2d27 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:34.483138 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de6eec5-90d6-4ee7-b660-e8a35dea2d27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:34.484592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de6eec5-90d6-4ee7-b660-e8a35dea2d27 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:34.493680 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 535/1063] 10.222.0.45 () {56 vars in 1115 bytes} [Wed Jun 15 14:30:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:34.502284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5c638-63d0-4d64-94ab-fb1504209a02 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:34.617112 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5c638-63d0-4d64-94ab-fb1504209a02 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:34.647513 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e036bfe-9a47-438e-aa2e-be46ef7c648f None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:34.647915 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e036bfe-9a47-438e-aa2e-be46ef7c648f None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:34.648091 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e036bfe-9a47-438e-aa2e-be46ef7c648f None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:34.653369 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae11cf-7830-4bb8-ae41-c79c70d4b42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:34.653544 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 536/1064] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:34.654060 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae11cf-7830-4bb8-ae41-c79c70d4b42d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:34.654344 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae11cf-7830-4bb8-ae41-c79c70d4b42d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:34.654716 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae11cf-7830-4bb8-ae41-c79c70d4b42d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:34.654829 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076b1dc-72a4-480a-988f-db958a21cc2c None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:34.655158 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076b1dc-72a4-480a-988f-db958a21cc2c None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:34.655397 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076b1dc-72a4-480a-988f-db958a21cc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:34.667516 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae11cf-7830-4bb8-ae41-c79c70d4b42d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:34.669827 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae11cf-7830-4bb8-ae41-c79c70d4b42d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:34.673321 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-4076b1dc-72a4-480a-988f-db958a21cc2c None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:34.674508 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-4076b1dc-72a4-480a-988f-db958a21cc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:34.693545 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-96ae11cf-7830-4bb8-ae41-c79c70d4b42d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:34.693545 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:34.693545 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:34.693545 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:34.693545 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:34.693545 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:34.693545 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:34.693545 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:34.693545 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:34.693545 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:34.693545 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:34.693545 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:34.693545 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:34.693545 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:34.693545 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:34.694489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:34.694489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:34.694489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:34.694489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:34.694489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:34.694489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:34.694489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:34.694489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:34.694489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:34.694489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:34.694489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:34.694489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:34.694489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:34.694489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:34.694489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:34.694489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:34.694489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:34.700327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:34.700327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:34.700327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:34.700327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:34.700327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 529/1065] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:34.700684 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae11cf-7830-4bb8-ae41-c79c70d4b42d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:34.765295 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:34.766479 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 537/1066] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:34.792810 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4076b1dc-72a4-480a-988f-db958a21cc2c None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:34.828816 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae11cf-7830-4bb8-ae41-c79c70d4b42d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:34.874615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed157db2-0fa1-46e1-817e-f39a23a58d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:34.875391 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed157db2-0fa1-46e1-817e-f39a23a58d7d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:34.882935 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed157db2-0fa1-46e1-817e-f39a23a58d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:34.882935 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed157db2-0fa1-46e1-817e-f39a23a58d7d None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:34.882935 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed157db2-0fa1-46e1-817e-f39a23a58d7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:34.882935 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed157db2-0fa1-46e1-817e-f39a23a58d7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:34.882935 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed157db2-0fa1-46e1-817e-f39a23a58d7d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:34.892598 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 530/1067] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:34.898210 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed157db2-0fa1-46e1-817e-f39a23a58d7d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:34.923448 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4076b1dc-72a4-480a-988f-db958a21cc2c None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:34.926131 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:34.983991 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9694512-a323-42e5-baf2-056fea7408bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:34.985404 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9694512-a323-42e5-baf2-056fea7408bb None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:34.985701 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9694512-a323-42e5-baf2-056fea7408bb None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:34.985985 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9694512-a323-42e5-baf2-056fea7408bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:35.000983 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9694512-a323-42e5-baf2-056fea7408bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:35.010776 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-e9694512-a323-42e5-baf2-056fea7408bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:35.010776 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:35.010776 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:35.010776 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:35.010776 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:35.010776 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:35.010776 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:35.010776 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:35.010776 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:35.010776 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:35.010776 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:35.010776 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:35.010776 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:35.010776 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:35.010776 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:35.010776 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:35.012589 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:35.012589 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:35.012589 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:35.012589 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:35.012589 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:35.012589 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:35.012589 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:35.012589 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:35.012589 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:35.012589 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:35.012589 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:35.012589 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:35.012589 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:35.012589 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:35.012589 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:35.012589 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:35.013450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:35.013450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:35.013450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:35.013450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:35.013450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:35.013450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:35.013450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:35.013450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:35.013450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:35.013450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:35.013450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:35.013450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:35.014110 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 538/1068] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:30:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:35.019395 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9694512-a323-42e5-baf2-056fea7408bb None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:35.028163 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed157db2-0fa1-46e1-817e-f39a23a58d7d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:35.038294 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:35.081205 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08241c9e-de6f-4cef-bc22-16fadef2c9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:35.082454 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08241c9e-de6f-4cef-bc22-16fadef2c9ff None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:35.082916 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08241c9e-de6f-4cef-bc22-16fadef2c9ff None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:35.083563 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08241c9e-de6f-4cef-bc22-16fadef2c9ff None admin] PATH_INFO: `/v3/users/nova` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:35.092300 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08241c9e-de6f-4cef-bc22-16fadef2c9ff None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:35.094011 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08241c9e-de6f-4cef-bc22-16fadef2c9ff None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:35.112224 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-08241c9e-de6f-4cef-bc22-16fadef2c9ff None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:30:35.112224 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:35.112224 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:35.112224 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:35.112224 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:35.112224 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:35.112224 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:35.112224 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:35.112224 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:35.112224 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:35.112224 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:35.112224 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:35.112224 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:30:35.112224 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:30:35.112224 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:30:35.112224 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:30:35.113423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:35.113423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:35.113423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:35.113423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:35.113423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:35.113423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:35.113423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:35.113423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:35.113423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:35.113423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:35.113423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:35.113423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:35.113423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:35.113423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:35.113423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:35.113423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:35.113423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:35.113423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:35.114355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:35.114355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:35.114355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:30:35.114355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:30:35.114355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:30:35.114355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:30:35.114355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:30:35.114355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:30:35.114355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:30:35.114355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:35.114878 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 531/1069] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Jun 15 14:30:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:35.119803 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08241c9e-de6f-4cef-bc22-16fadef2c9ff None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:35.151570 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9694512-a323-42e5-baf2-056fea7408bb None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:35.198133 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa857cc1-cdd7-4cdd-9b42-6f27357ed2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:35.199753 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa857cc1-cdd7-4cdd-9b42-6f27357ed2ea None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:35.200257 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa857cc1-cdd7-4cdd-9b42-6f27357ed2ea None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:35.200720 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa857cc1-cdd7-4cdd-9b42-6f27357ed2ea None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:35.201791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa857cc1-cdd7-4cdd-9b42-6f27357ed2ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:35.202312 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa857cc1-cdd7-4cdd-9b42-6f27357ed2ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:35.204029 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa857cc1-cdd7-4cdd-9b42-6f27357ed2ea None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:35.219102 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 539/1070] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:35.220197 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa857cc1-cdd7-4cdd-9b42-6f27357ed2ea None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:35.255684 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08241c9e-de6f-4cef-bc22-16fadef2c9ff None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:35.260500 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:35.306537 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd170b4-0ec9-44b1-9c14-bb42801c398a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:35.308119 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd170b4-0ec9-44b1-9c14-bb42801c398a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:35.308614 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd170b4-0ec9-44b1-9c14-bb42801c398a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:35.309065 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd170b4-0ec9-44b1-9c14-bb42801c398a None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:35.310238 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd170b4-0ec9-44b1-9c14-bb42801c398a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:35.311013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd170b4-0ec9-44b1-9c14-bb42801c398a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:35.312538 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd170b4-0ec9-44b1-9c14-bb42801c398a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:35.336875 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 532/1071] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Jun 15 14:30:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:35.343369 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd170b4-0ec9-44b1-9c14-bb42801c398a None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:35.354536 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa857cc1-cdd7-4cdd-9b42-6f27357ed2ea None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:35.371840 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:35.399346 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e19b787-ad69-4f5a-9577-19c520ed1c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:35.400511 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19b787-ad69-4f5a-9577-19c520ed1c26 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:35.400799 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19b787-ad69-4f5a-9577-19c520ed1c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:35.401146 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19b787-ad69-4f5a-9577-19c520ed1c26 None admin] PATH_INFO: `/v3/users/614f8cdc30784b39b1e8370fa40186a6` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:35.403090 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e19b787-ad69-4f5a-9577-19c520ed1c26 None admin] RBAC: Authorizing `identity:get_user(user_id=614f8cdc30784b39b1e8370fa40186a6)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:35.404553 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e19b787-ad69-4f5a-9577-19c520ed1c26 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:35.407044 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 540/1072] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:35 2022] GET /identity/v3/users/614f8cdc30784b39b1e8370fa40186a6 => generated 285 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:35.411929 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e19b787-ad69-4f5a-9577-19c520ed1c26 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:35.463918 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd170b4-0ec9-44b1-9c14-bb42801c398a None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:35.506361 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f8e81a-c173-4e49-a763-0c9d14a95547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:35.507139 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f8e81a-c173-4e49-a763-0c9d14a95547 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:35.507424 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f8e81a-c173-4e49-a763-0c9d14a95547 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:35.507695 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f8e81a-c173-4e49-a763-0c9d14a95547 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:35.519993 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f8e81a-c173-4e49-a763-0c9d14a95547 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:35.522207 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f8e81a-c173-4e49-a763-0c9d14a95547 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:35.539335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:35.541798 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e19b787-ad69-4f5a-9577-19c520ed1c26 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:35.542401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-d4f8e81a-c173-4e49-a763-0c9d14a95547 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:35.542401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:35.542401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:35.542401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:35.542401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:35.542401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:35.542401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:35.542401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:35.542401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:35.542401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:35.542401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:35.542401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:35.542401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:35.542401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:35.542401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:35.544006 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:35.544006 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:35.544006 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:35.544006 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:35.544006 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:35.544006 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:35.544006 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:35.544006 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:35.544006 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:35.544006 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:35.544006 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:35.544006 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:35.544006 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:35.544006 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:35.544006 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:35.544006 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:35.544006 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:35.545091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:35.545091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:35.545091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:35.545091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:35.545091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 533/1073] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:35.550104 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f8e81a-c173-4e49-a763-0c9d14a95547 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:35.585561 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe076d2-0590-450f-9783-02c38e9849a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:35.586458 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe076d2-0590-450f-9783-02c38e9849a6 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:35.586890 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe076d2-0590-450f-9783-02c38e9849a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:35.587199 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe076d2-0590-450f-9783-02c38e9849a6 None admin] PATH_INFO: `/v3/projects/266b416e205a46aa8c934fbcae0f571d` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:35.588938 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe076d2-0590-450f-9783-02c38e9849a6 None admin] RBAC: Authorizing `identity:get_project(project_id=266b416e205a46aa8c934fbcae0f571d)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:35.591162 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe076d2-0590-450f-9783-02c38e9849a6 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:35.599033 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 541/1074] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:30:35 2022] GET /identity/v3/projects/266b416e205a46aa8c934fbcae0f571d => generated 311 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:35.603085 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe076d2-0590-450f-9783-02c38e9849a6 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:35.648393 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:35.681844 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f8e81a-c173-4e49-a763-0c9d14a95547 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:35.734589 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959da1b7-767c-411e-b0af-a77dcf8fb39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:35.735431 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959da1b7-767c-411e-b0af-a77dcf8fb39e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:35.735716 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959da1b7-767c-411e-b0af-a77dcf8fb39e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:35.735993 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959da1b7-767c-411e-b0af-a77dcf8fb39e None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:35.736721 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959da1b7-767c-411e-b0af-a77dcf8fb39e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:35.739029 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959da1b7-767c-411e-b0af-a77dcf8fb39e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:35.739029 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959da1b7-767c-411e-b0af-a77dcf8fb39e None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:35.747577 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe076d2-0590-450f-9783-02c38e9849a6 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:35.758069 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 534/1075] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:35.758669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959da1b7-767c-411e-b0af-a77dcf8fb39e None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:35.786210 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7dbe4-e186-4c3f-ad9e-b72fbad8b7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:35.787302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7dbe4-e186-4c3f-ad9e-b72fbad8b7d5 None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:35.787412 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7dbe4-e186-4c3f-ad9e-b72fbad8b7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:35.789086 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7dbe4-e186-4c3f-ad9e-b72fbad8b7d5 None admin] PATH_INFO: `/v3/projects/266b416e205a46aa8c934fbcae0f571d/users/614f8cdc30784b39b1e8370fa40186a6/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:35.790857 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c7dbe4-e186-4c3f-ad9e-b72fbad8b7d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=266b416e205a46aa8c934fbcae0f571d, user_id=614f8cdc30784b39b1e8370fa40186a6, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:35.792447 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c7dbe4-e186-4c3f-ad9e-b72fbad8b7d5 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:35.814468 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 542/1076] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Jun 15 14:30:35 2022] PUT /identity/v3/projects/266b416e205a46aa8c934fbcae0f571d/users/614f8cdc30784b39b1e8370fa40186a6/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:30:35.900160 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959da1b7-767c-411e-b0af-a77dcf8fb39e None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:35.941684 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2014636d-3153-4c5f-9e9d-6abdaf88067b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:35.942817 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014636d-3153-4c5f-9e9d-6abdaf88067b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:35.943296 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014636d-3153-4c5f-9e9d-6abdaf88067b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:35.943746 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014636d-3153-4c5f-9e9d-6abdaf88067b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:35.947337 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:35.955355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2014636d-3153-4c5f-9e9d-6abdaf88067b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:35.957176 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2014636d-3153-4c5f-9e9d-6abdaf88067b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:35.975098 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-2014636d-3153-4c5f-9e9d-6abdaf88067b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:35.975098 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:35.975098 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:35.975098 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:35.975098 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:35.975098 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:35.975098 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:35.975098 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:35.975098 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:35.975098 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:35.975098 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:35.975098 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:35.975098 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:30:35.975098 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:30:35.975098 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:30:35.976047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:30:35.976047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:35.976047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:35.976047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:35.976047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:35.976047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:35.976047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:35.976047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:35.976047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:35.976047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:35.976047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:35.976047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:35.976047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:35.976047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:35.976047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:35.976047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:35.976047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:35.976937 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:30:35.976937 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:30:35.976937 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:30:35.976937 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:30:35.976937 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:30:35.976937 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:30:35.976937 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:35.976937 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:35.979517 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 535/1077] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:35.986983 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7dbe4-e186-4c3f-ad9e-b72fbad8b7d5 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:36.017695 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:36.108662 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7dbe4-e186-4c3f-ad9e-b72fbad8b7d5 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:36.145929 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d57a6-800b-4160-bb76-9a9c35b226d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:36.147221 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d57a6-800b-4160-bb76-9a9c35b226d1 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:36.147692 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d57a6-800b-4160-bb76-9a9c35b226d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:36.148150 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d57a6-800b-4160-bb76-9a9c35b226d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:36.149126 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d57a6-800b-4160-bb76-9a9c35b226d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:36.149731 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d57a6-800b-4160-bb76-9a9c35b226d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:36.151402 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d57a6-800b-4160-bb76-9a9c35b226d1 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:36.165009 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 543/1078] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:30:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:36.172156 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2014636d-3153-4c5f-9e9d-6abdaf88067b None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:36.292741 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2014636d-3153-4c5f-9e9d-6abdaf88067b None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:36.333302 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962dd964-06b2-427c-9313-a211a3222948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:36.335132 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962dd964-06b2-427c-9313-a211a3222948 None admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:36.335658 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962dd964-06b2-427c-9313-a211a3222948 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:36.336162 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962dd964-06b2-427c-9313-a211a3222948 None admin] PATH_INFO: `/v3/projects/d488c79cb1704e9496fc196d8aa6c104/users/f3185bd2458c4e22830975700888e9be/roles/48dafbec535e4eee8263719557d59d6f` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:36.339025 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962dd964-06b2-427c-9313-a211a3222948 None admin] RBAC: Authorizing `identity:create_grant(project_id=d488c79cb1704e9496fc196d8aa6c104, user_id=f3185bd2458c4e22830975700888e9be, role_id=48dafbec535e4eee8263719557d59d6f)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:36.341389 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962dd964-06b2-427c-9313-a211a3222948 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:36.365887 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 536/1079] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Jun 15 14:30:36 2022] PUT /identity/v3/projects/d488c79cb1704e9496fc196d8aa6c104/users/f3185bd2458c4e22830975700888e9be/roles/48dafbec535e4eee8263719557d59d6f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:30:36.416244 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce6cc1c-bb4d-4359-9e62-5b1a231e7a07 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:36.416739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce6cc1c-bb4d-4359-9e62-5b1a231e7a07 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:36.416926 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce6cc1c-bb4d-4359-9e62-5b1a231e7a07 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:36.418390 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 544/1080] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:36.424248 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06fbb25-15bb-46d0-bdc5-adab494e0eec None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:36.424688 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06fbb25-15bb-46d0-bdc5-adab494e0eec None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:36.425086 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06fbb25-15bb-46d0-bdc5-adab494e0eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:36.446432 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-a06fbb25-15bb-46d0-bdc5-adab494e0eec None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:36.448042 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-a06fbb25-15bb-46d0-bdc5-adab494e0eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:36.520861 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:36.539588 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 537/1081] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:36.564404 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce6cc1c-bb4d-4359-9e62-5b1a231e7a07 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:36.657693 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:36.683086 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce6cc1c-bb4d-4359-9e62-5b1a231e7a07 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:36.728353 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8fb600-dcda-49f2-be43-b5df399a7316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:36.728597 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fb600-dcda-49f2-be43-b5df399a7316 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:36.728890 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fb600-dcda-49f2-be43-b5df399a7316 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:36.729167 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fb600-dcda-49f2-be43-b5df399a7316 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:36.807216 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8fb600-dcda-49f2-be43-b5df399a7316 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:36.809486 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8fb600-dcda-49f2-be43-b5df399a7316 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:36.910288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:36.942105 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 545/1082] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:30:36.944092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06fbb25-15bb-46d0-bdc5-adab494e0eec None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:37.054675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06fbb25-15bb-46d0-bdc5-adab494e0eec None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:37.091495 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394448f1-b99e-407f-8110-be67622d3b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:37.092293 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394448f1-b99e-407f-8110-be67622d3b00 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:37.092575 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394448f1-b99e-407f-8110-be67622d3b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:37.092848 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394448f1-b99e-407f-8110-be67622d3b00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:37.103775 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394448f1-b99e-407f-8110-be67622d3b00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:37.105348 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394448f1-b99e-407f-8110-be67622d3b00 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:37.123964 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-394448f1-b99e-407f-8110-be67622d3b00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:37.123964 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:37.123964 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:37.123964 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:37.123964 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:37.123964 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:37.123964 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:37.123964 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:37.123964 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:37.123964 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:37.123964 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:37.123964 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:37.123964 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:37.123964 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:37.123964 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:37.124887 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:37.124887 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:37.124887 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:37.124887 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:37.124887 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:37.124887 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:37.124887 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:37.124887 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:37.124887 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:37.124887 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:37.124887 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:37.124887 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:37.124887 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:37.124887 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:37.124887 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:37.124887 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:37.124887 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:37.125758 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:37.125758 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:37.125758 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:37.125758 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:37.133294 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 538/1083] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:37.133449 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8fb600-dcda-49f2-be43-b5df399a7316 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:37.248151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8fb600-dcda-49f2-be43-b5df399a7316 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:37.288420 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef478d-4096-4d2e-b223-f85e775ef063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:37.289093 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef478d-4096-4d2e-b223-f85e775ef063 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:37.289385 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef478d-4096-4d2e-b223-f85e775ef063 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:37.289649 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef478d-4096-4d2e-b223-f85e775ef063 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:37.290370 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caef478d-4096-4d2e-b223-f85e775ef063 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:37.290677 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caef478d-4096-4d2e-b223-f85e775ef063 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:37.292204 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caef478d-4096-4d2e-b223-f85e775ef063 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:37.311425 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 546/1084] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:37.988777 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c92af-408c-4d9d-a663-2f37136c1ed9 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:37.989238 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c92af-408c-4d9d-a663-2f37136c1ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:37.989238 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c92af-408c-4d9d-a663-2f37136c1ed9 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:37.990615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 539/1085] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:30:37.995899 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1781d18-ab14-4ac8-b26e-81bd50633ff6 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:37.996338 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1781d18-ab14-4ac8-b26e-81bd50633ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:37.996498 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1781d18-ab14-4ac8-b26e-81bd50633ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:38.014866 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-a1781d18-ab14-4ac8-b26e-81bd50633ff6 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:38.016167 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a1781d18-ab14-4ac8-b26e-81bd50633ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:38.052325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:38.086266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 547/1086] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:38.109321 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c92af-408c-4d9d-a663-2f37136c1ed9 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:38.157878 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:38.220495 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c92af-408c-4d9d-a663-2f37136c1ed9 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:38.254695 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb556a-2be7-4441-ab1a-29cf79afee0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:38.255511 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb556a-2be7-4441-ab1a-29cf79afee0e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:38.255797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb556a-2be7-4441-ab1a-29cf79afee0e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:38.256074 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb556a-2be7-4441-ab1a-29cf79afee0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:38.266075 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bb556a-2be7-4441-ab1a-29cf79afee0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:38.273839 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-92bb556a-2be7-4441-ab1a-29cf79afee0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:38.273839 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:38.273839 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:38.273839 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:38.273839 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:38.273839 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:38.273839 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:38.273839 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:38.273839 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:38.273839 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:38.273839 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:38.273839 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:38.273839 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:38.273839 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:38.273839 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:38.273839 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:38.274726 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:38.274726 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:38.274726 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:38.274726 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:38.274726 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:38.274726 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:38.274726 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:38.274726 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:38.274726 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:38.274726 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:38.274726 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:38.274726 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:38.274726 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:38.274726 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:38.274726 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:38.274726 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:38.275691 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:38.275691 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:38.275691 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:38.275691 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:38.275691 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:38.275691 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:38.275691 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:38.275691 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:38.275691 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:38.275691 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:38.275691 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:38.275691 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:38.280710 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 540/1087] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:30:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:38.283069 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1781d18-ab14-4ac8-b26e-81bd50633ff6 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:38.393402 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1781d18-ab14-4ac8-b26e-81bd50633ff6 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:38.427533 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9d459-2a5a-4d15-b989-2c92f9be7a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:38.428199 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9d459-2a5a-4d15-b989-2c92f9be7a4c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:38.428478 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9d459-2a5a-4d15-b989-2c92f9be7a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:38.428750 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9d459-2a5a-4d15-b989-2c92f9be7a4c None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:38.429485 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9d459-2a5a-4d15-b989-2c92f9be7a4c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:38.429787 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9d459-2a5a-4d15-b989-2c92f9be7a4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:38.431191 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9d459-2a5a-4d15-b989-2c92f9be7a4c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:38.439915 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 548/1088] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:38.445659 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb556a-2be7-4441-ab1a-29cf79afee0e None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:38.524321 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe32b1-94a6-45a6-b356-01e9d34f1d8e None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:38.524534 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe32b1-94a6-45a6-b356-01e9d34f1d8e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:38.524805 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe32b1-94a6-45a6-b356-01e9d34f1d8e None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:38.526181 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 549/1089] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:30:38.531505 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a813584-fe06-4bad-bd33-ea2ad9fe1ccf None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:38.531787 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a813584-fe06-4bad-bd33-ea2ad9fe1ccf None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:38.532056 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a813584-fe06-4bad-bd33-ea2ad9fe1ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:38.549080 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-4a813584-fe06-4bad-bd33-ea2ad9fe1ccf None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:38.550255 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-4a813584-fe06-4bad-bd33-ea2ad9fe1ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:38.565950 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:38.570037 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb556a-2be7-4441-ab1a-29cf79afee0e None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:38.584073 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:38.608957 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ddaca-f018-4472-b75e-748f47633bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:38.609832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ddaca-f018-4472-b75e-748f47633bfe None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:38.610116 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ddaca-f018-4472-b75e-748f47633bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:38.610808 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ddaca-f018-4472-b75e-748f47633bfe None admin] PATH_INFO: `/v3/users/614f8cdc30784b39b1e8370fa40186a6` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:38.611756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93ddaca-f018-4472-b75e-748f47633bfe None admin] RBAC: Authorizing `identity:get_user(user_id=614f8cdc30784b39b1e8370fa40186a6)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:38.613308 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93ddaca-f018-4472-b75e-748f47633bfe None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:38.614824 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 541/1090] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:38 2022] GET /identity/v3/users/614f8cdc30784b39b1e8370fa40186a6 => generated 285 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:38.622324 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ddaca-f018-4472-b75e-748f47633bfe None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:38.630671 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 550/1091] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:30:38.654480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a813584-fe06-4bad-bd33-ea2ad9fe1ccf None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:38.731338 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ddaca-f018-4472-b75e-748f47633bfe None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:38.760108 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a813584-fe06-4bad-bd33-ea2ad9fe1ccf None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:38.766677 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb96b080-4d83-4a1f-b4e7-80fe6782f07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:38.767929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96b080-4d83-4a1f-b4e7-80fe6782f07e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:38.768214 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96b080-4d83-4a1f-b4e7-80fe6782f07e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:38.768494 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96b080-4d83-4a1f-b4e7-80fe6782f07e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:38.796872 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b4ba2-dce5-41e2-992c-39941435a152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:38.797664 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b4ba2-dce5-41e2-992c-39941435a152 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:38.797948 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b4ba2-dce5-41e2-992c-39941435a152 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:38.798218 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b4ba2-dce5-41e2-992c-39941435a152 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:38.806661 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:38.808884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b4ba2-dce5-41e2-992c-39941435a152 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:38.817039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-4a0b4ba2-dce5-41e2-992c-39941435a152 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:30:38.817039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:38.817039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:38.817039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:38.817039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:38.817039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:38.817039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:38.817039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:38.817039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:38.817039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:38.817039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:38.817039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:38.817039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:38.817039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:38.817039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:38.817039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:38.818159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:38.818159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:38.818159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:38.818159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:38.818159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:38.818159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:38.818159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:38.818159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:38.818159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:38.818159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:38.818159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:38.818159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:38.818159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:38.818159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:38.818159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:38.818159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:38.819260 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:38.819260 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:38.819260 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:38.819260 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:38.819260 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:38.819260 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:38.819260 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:38.819260 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:38.819260 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:38.819260 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:38.819260 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:30:38.819260 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:38.819260 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 551/1092] 10.222.0.45 () {56 vars in 1120 bytes} [Wed Jun 15 14:30:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:38.824031 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b4ba2-dce5-41e2-992c-39941435a152 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:38.850938 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:38.853472 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb96b080-4d83-4a1f-b4e7-80fe6782f07e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:38.854918 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb96b080-4d83-4a1f-b4e7-80fe6782f07e None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:38.932706 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b4ba2-dce5-41e2-992c-39941435a152 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:38.957596 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 542/1093] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:30:38.964167 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb96b080-4d83-4a1f-b4e7-80fe6782f07e None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:38.969481 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1909e7-8488-41a8-ae18-a5365d662120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:38.970379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1909e7-8488-41a8-ae18-a5365d662120 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:38.970807 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1909e7-8488-41a8-ae18-a5365d662120 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:38.971261 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1909e7-8488-41a8-ae18-a5365d662120 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:38.972225 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1909e7-8488-41a8-ae18-a5365d662120 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:38.972539 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1909e7-8488-41a8-ae18-a5365d662120 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:38.973956 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1909e7-8488-41a8-ae18-a5365d662120 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:38.982340 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 552/1094] 10.222.0.45 () {56 vars in 1115 bytes} [Wed Jun 15 14:30:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:38.988472 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1909e7-8488-41a8-ae18-a5365d662120 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:39.035130 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:39.073865 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb96b080-4d83-4a1f-b4e7-80fe6782f07e None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:39.082190 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:39.098241 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1909e7-8488-41a8-ae18-a5365d662120 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:39.108940 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c996fc-57f0-4b29-8d54-2ee101ee9c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:39.109827 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c996fc-57f0-4b29-8d54-2ee101ee9c6e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:39.110112 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c996fc-57f0-4b29-8d54-2ee101ee9c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:39.110386 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c996fc-57f0-4b29-8d54-2ee101ee9c6e None admin] PATH_INFO: `/v3/projects/266b416e205a46aa8c934fbcae0f571d` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:39.111587 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c996fc-57f0-4b29-8d54-2ee101ee9c6e None admin] RBAC: Authorizing `identity:get_project(project_id=266b416e205a46aa8c934fbcae0f571d)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:39.113023 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c996fc-57f0-4b29-8d54-2ee101ee9c6e None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:39.114712 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 543/1095] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:30:38 2022] GET /identity/v3/projects/266b416e205a46aa8c934fbcae0f571d => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:39.121179 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c996fc-57f0-4b29-8d54-2ee101ee9c6e None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:39.134363 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4081d-5be8-4437-afc9-46eb750678bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:39.135191 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4081d-5be8-4437-afc9-46eb750678bf None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:39.135470 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4081d-5be8-4437-afc9-46eb750678bf None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:39.135750 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4081d-5be8-4437-afc9-46eb750678bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:39.146448 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa4081d-5be8-4437-afc9-46eb750678bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:39.148293 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa4081d-5be8-4437-afc9-46eb750678bf None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:39.165378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-0aa4081d-5be8-4437-afc9-46eb750678bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:39.165378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:39.165378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:39.165378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:39.165378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:39.165378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:39.165378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:39.165378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:39.165378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:39.165378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:39.165378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:39.165378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:39.165378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:39.165378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:39.165378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:39.166463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:39.166463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:39.166463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:39.166463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:39.166463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:39.166463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:39.166463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:39.166463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:39.166463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:39.166463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:39.166463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:39.166463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:39.166463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:39.166463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:39.166463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:39.166463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:39.166463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:39.167479 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:39.167479 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:39.167479 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:39.167479 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:39.167479 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 553/1096] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:39.172666 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4081d-5be8-4437-afc9-46eb750678bf None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:39.224131 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c996fc-57f0-4b29-8d54-2ee101ee9c6e None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:39.259592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc8c0b0-2f2d-4351-9de7-6696787809c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:39.260326 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc8c0b0-2f2d-4351-9de7-6696787809c9 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:39.260611 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc8c0b0-2f2d-4351-9de7-6696787809c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:39.260884 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc8c0b0-2f2d-4351-9de7-6696787809c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:39.261726 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc8c0b0-2f2d-4351-9de7-6696787809c9 None admin] RBAC: Adding query filter params (role.id=ed1cfa2fcf724a4d84f5acea100930c7, scope.project.id=266b416e205a46aa8c934fbcae0f571d, user.id=614f8cdc30784b39b1e8370fa40186a6) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:39.262037 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc8c0b0-2f2d-4351-9de7-6696787809c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:39.263480 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc8c0b0-2f2d-4351-9de7-6696787809c9 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:39.266674 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:39.279395 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 544/1097] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:39 2022] GET /identity/v3/role_assignments?user.id=614f8cdc30784b39b1e8370fa40186a6&scope.project.id=266b416e205a46aa8c934fbcae0f571d&role.id=ed1cfa2fcf724a4d84f5acea100930c7 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:39.280523 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4081d-5be8-4437-afc9-46eb750678bf None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:39.326975 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb30534-56e0-45eb-87fd-d41b52bf6e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:39.327624 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb30534-56e0-45eb-87fd-d41b52bf6e1a None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:39.327912 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb30534-56e0-45eb-87fd-d41b52bf6e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:39.328187 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb30534-56e0-45eb-87fd-d41b52bf6e1a None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:39.328939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb30534-56e0-45eb-87fd-d41b52bf6e1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:39.329251 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb30534-56e0-45eb-87fd-d41b52bf6e1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:39.330665 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb30534-56e0-45eb-87fd-d41b52bf6e1a None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:39.338568 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:39.346307 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 554/1098] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:39.351914 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc8c0b0-2f2d-4351-9de7-6696787809c9 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:39.459576 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc8c0b0-2f2d-4351-9de7-6696787809c9 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:39.497851 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e76d7-9cd4-4075-9de8-98e7d750bfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:39.498986 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e76d7-9cd4-4075-9de8-98e7d750bfca None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:39.499664 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e76d7-9cd4-4075-9de8-98e7d750bfca None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:39.499664 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e76d7-9cd4-4075-9de8-98e7d750bfca None admin] PATH_INFO: `/v3/users/nova` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:39.507945 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350e76d7-9cd4-4075-9de8-98e7d750bfca None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:39.508793 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350e76d7-9cd4-4075-9de8-98e7d750bfca None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:39.524539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-350e76d7-9cd4-4075-9de8-98e7d750bfca None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:30:39.524539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:39.524539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:39.524539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:39.524539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:39.524539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:39.524539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:39.524539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:39.524539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:39.524539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:39.524539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:39.524539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:39.524539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:30:39.524539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:30:39.524539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:30:39.524539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:30:39.525708 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:39.525708 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:39.525708 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:39.525708 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:39.525708 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:39.525708 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:39.525708 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:39.525708 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:39.525708 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:39.525708 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:39.525708 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:39.525708 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:39.525708 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:39.525708 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:39.525708 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:39.525708 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:39.525708 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:39.525708 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:39.526678 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:39.526678 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:39.526678 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:30:39.526678 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:30:39.526678 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:30:39.526678 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:30:39.526678 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:30:39.526678 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:30:39.526678 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:30:39.526678 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:39.526678 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 545/1099] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Jun 15 14:30:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:39.532651 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb30534-56e0-45eb-87fd-d41b52bf6e1a None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:39.639264 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb30534-56e0-45eb-87fd-d41b52bf6e1a None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:39.678466 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b404f5-1bf0-4c0c-a093-13c9992f6ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:39.679180 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b404f5-1bf0-4c0c-a093-13c9992f6ec3 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:39.679465 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b404f5-1bf0-4c0c-a093-13c9992f6ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:39.679739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b404f5-1bf0-4c0c-a093-13c9992f6ec3 None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:39.680588 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b404f5-1bf0-4c0c-a093-13c9992f6ec3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:39.680898 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b404f5-1bf0-4c0c-a093-13c9992f6ec3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:39.682333 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b404f5-1bf0-4c0c-a093-13c9992f6ec3 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:39.701296 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 555/1100] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Jun 15 14:30:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:39.707650 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e76d7-9cd4-4075-9de8-98e7d750bfca None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:39.776614 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:39.826599 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e76d7-9cd4-4075-9de8-98e7d750bfca None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:39.863870 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe00d95-949e-4403-aa9f-a6ad36a636e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:39.864978 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe00d95-949e-4403-aa9f-a6ad36a636e7 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:39.865484 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe00d95-949e-4403-aa9f-a6ad36a636e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:39.865931 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe00d95-949e-4403-aa9f-a6ad36a636e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:39.945312 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe00d95-949e-4403-aa9f-a6ad36a636e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:39.946692 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe00d95-949e-4403-aa9f-a6ad36a636e7 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:40.008062 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:40.053287 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 546/1101] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:30:40.059689 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b404f5-1bf0-4c0c-a093-13c9992f6ec3 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:40.117465 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:40.165449 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b404f5-1bf0-4c0c-a093-13c9992f6ec3 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:40.200415 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1461e592-39d2-49fb-b7f9-987d2abaafee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:40.201122 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461e592-39d2-49fb-b7f9-987d2abaafee None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:40.201402 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461e592-39d2-49fb-b7f9-987d2abaafee None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:40.201671 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461e592-39d2-49fb-b7f9-987d2abaafee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:40.212030 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1461e592-39d2-49fb-b7f9-987d2abaafee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:40.213644 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1461e592-39d2-49fb-b7f9-987d2abaafee None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:40.231918 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-1461e592-39d2-49fb-b7f9-987d2abaafee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:40.231918 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:40.231918 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:40.231918 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:40.231918 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:40.231918 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:40.231918 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:40.231918 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:40.231918 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:40.231918 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:40.231918 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:40.231918 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:40.231918 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:40.231918 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:40.231918 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:40.232911 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:40.232911 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:40.232911 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:40.232911 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:40.232911 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:40.232911 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:40.232911 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:40.232911 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:40.232911 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:40.232911 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:40.232911 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:40.232911 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:40.232911 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:40.232911 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:40.232911 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:40.232911 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:40.232911 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:40.233848 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:40.233848 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:40.233848 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:40.233848 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:40.233848 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 556/1102] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:40.239196 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe00d95-949e-4403-aa9f-a6ad36a636e7 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:40.349776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe00d95-949e-4403-aa9f-a6ad36a636e7 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:40.385289 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80804c-d62a-4ca9-85ff-1a20d5f9d64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:40.386298 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80804c-d62a-4ca9-85ff-1a20d5f9d64c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:40.387266 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80804c-d62a-4ca9-85ff-1a20d5f9d64c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:40.387544 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80804c-d62a-4ca9-85ff-1a20d5f9d64c None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:40.388272 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d80804c-d62a-4ca9-85ff-1a20d5f9d64c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:40.388628 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d80804c-d62a-4ca9-85ff-1a20d5f9d64c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:40.390032 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d80804c-d62a-4ca9-85ff-1a20d5f9d64c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:40.404569 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 547/1103] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:40.410254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1461e592-39d2-49fb-b7f9-987d2abaafee None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:40.518334 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1461e592-39d2-49fb-b7f9-987d2abaafee None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:40.529249 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:40.556076 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64df168-f1aa-4714-87f2-9e13da288527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:40.557216 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64df168-f1aa-4714-87f2-9e13da288527 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:40.557502 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64df168-f1aa-4714-87f2-9e13da288527 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:40.557781 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64df168-f1aa-4714-87f2-9e13da288527 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:40.565117 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64df168-f1aa-4714-87f2-9e13da288527 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:40.566584 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64df168-f1aa-4714-87f2-9e13da288527 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:40.580870 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-a64df168-f1aa-4714-87f2-9e13da288527 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:40.580870 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:40.580870 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:40.580870 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:40.580870 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:40.580870 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:40.580870 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:40.580870 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:40.580870 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:40.580870 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:40.580870 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:40.580870 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:40.580870 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:30:40.580870 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:30:40.580870 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:30:40.581769 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:30:40.581769 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:40.581769 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:40.581769 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:40.581769 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:40.581769 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:40.581769 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:40.581769 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:40.581769 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:40.581769 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:40.581769 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:40.581769 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:40.581769 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:40.581769 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:40.581769 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:40.581769 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:40.581769 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:40.582664 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:30:40.582664 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:30:40.582664 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:30:40.582664 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:30:40.582664 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:30:40.582664 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:30:40.582664 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:40.582664 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:30:40.588383 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 557/1104] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:40.588521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80804c-d62a-4ca9-85ff-1a20d5f9d64c None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:40.614858 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:40.698401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80804c-d62a-4ca9-85ff-1a20d5f9d64c None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:40.734464 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6639c7e-3fb4-4c2d-9641-47b845ff2f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:40.735166 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6639c7e-3fb4-4c2d-9641-47b845ff2f92 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:40.735453 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6639c7e-3fb4-4c2d-9641-47b845ff2f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:40.735725 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6639c7e-3fb4-4c2d-9641-47b845ff2f92 None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:40.736500 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6639c7e-3fb4-4c2d-9641-47b845ff2f92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:40.736810 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6639c7e-3fb4-4c2d-9641-47b845ff2f92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:40.738227 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6639c7e-3fb4-4c2d-9641-47b845ff2f92 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:40.750596 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 548/1105] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:30:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:40.756885 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64df168-f1aa-4714-87f2-9e13da288527 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:40.858007 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64df168-f1aa-4714-87f2-9e13da288527 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:40.891001 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c2b05-de28-47f8-a363-fb56bc04c1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:40.891656 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c2b05-de28-47f8-a363-fb56bc04c1b5 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:40.891937 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c2b05-de28-47f8-a363-fb56bc04c1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:40.892208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c2b05-de28-47f8-a363-fb56bc04c1b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:40.893031 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35c2b05-de28-47f8-a363-fb56bc04c1b5 None admin] RBAC: Adding query filter params (role.id=48dafbec535e4eee8263719557d59d6f, scope.project.id=d488c79cb1704e9496fc196d8aa6c104, user.id=f3185bd2458c4e22830975700888e9be) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:40.893340 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35c2b05-de28-47f8-a363-fb56bc04c1b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:40.894949 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35c2b05-de28-47f8-a363-fb56bc04c1b5 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:40.901269 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 558/1106] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:40 2022] GET /identity/v3/role_assignments?user.id=f3185bd2458c4e22830975700888e9be&scope.project.id=d488c79cb1704e9496fc196d8aa6c104&role.id=48dafbec535e4eee8263719557d59d6f => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:41.440377 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bea1cc-246d-41a6-afb7-e39f4a8fe04f None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:41.440888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bea1cc-246d-41a6-afb7-e39f4a8fe04f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:41.441009 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bea1cc-246d-41a6-afb7-e39f4a8fe04f None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:41.442305 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 549/1107] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:41.447699 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dcf3c1-7055-47e6-ba86-32a6b87f4f3a None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:41.448124 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dcf3c1-7055-47e6-ba86-32a6b87f4f3a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:41.448259 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dcf3c1-7055-47e6-ba86-32a6b87f4f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:41.466987 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-64dcf3c1-7055-47e6-ba86-32a6b87f4f3a None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:41.468203 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-64dcf3c1-7055-47e6-ba86-32a6b87f4f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:41.494061 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:41.537626 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 559/1108] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:41.565712 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bea1cc-246d-41a6-afb7-e39f4a8fe04f None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:41.647930 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:41.673077 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bea1cc-246d-41a6-afb7-e39f4a8fe04f None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:41.709215 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb1508-8d09-432e-8321-f986bdbf702e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:41.709948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb1508-8d09-432e-8321-f986bdbf702e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:41.710301 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb1508-8d09-432e-8321-f986bdbf702e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:41.710586 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb1508-8d09-432e-8321-f986bdbf702e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:41.711861 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cb1508-8d09-432e-8321-f986bdbf702e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:41.713311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cb1508-8d09-432e-8321-f986bdbf702e None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:41.714971 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 550/1109] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:41.720928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dcf3c1-7055-47e6-ba86-32a6b87f4f3a None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:41.818635 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dcf3c1-7055-47e6-ba86-32a6b87f4f3a None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:41.850919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22dbcc1-a3b3-48d2-9560-e217953d090b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:41.851601 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22dbcc1-a3b3-48d2-9560-e217953d090b None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:41.851881 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22dbcc1-a3b3-48d2-9560-e217953d090b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:41.852151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22dbcc1-a3b3-48d2-9560-e217953d090b None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:41.852854 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22dbcc1-a3b3-48d2-9560-e217953d090b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:41.854272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22dbcc1-a3b3-48d2-9560-e217953d090b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:41.874016 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 560/1110] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Jun 15 14:30:41 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:44.007928 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24c976-2304-46a0-b714-fc1e2eef8c33 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:44.008440 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24c976-2304-46a0-b714-fc1e2eef8c33 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:44.008440 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24c976-2304-46a0-b714-fc1e2eef8c33 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:44.009769 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 551/1111] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:44.015462 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da80537-4b89-4058-9b55-4bf452f110cd None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:44.015892 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da80537-4b89-4058-9b55-4bf452f110cd None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:44.016004 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da80537-4b89-4058-9b55-4bf452f110cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:44.032650 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-1da80537-4b89-4058-9b55-4bf452f110cd None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:44.033832 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1da80537-4b89-4058-9b55-4bf452f110cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:44.046601 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:44.099264 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 561/1112] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:44.122515 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24c976-2304-46a0-b714-fc1e2eef8c33 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:44.220674 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24c976-2304-46a0-b714-fc1e2eef8c33 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:44.253763 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8879f8f-732c-4cae-8393-d2e11bf368a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:44.254604 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8879f8f-732c-4cae-8393-d2e11bf368a2 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:44.254892 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8879f8f-732c-4cae-8393-d2e11bf368a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:44.255197 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8879f8f-732c-4cae-8393-d2e11bf368a2 None admin] PATH_INFO: `/v3/roles/6b6353b49c6a41b1a157f8997fc52d9a` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:44.256658 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8879f8f-732c-4cae-8393-d2e11bf368a2 None admin] RBAC: Authorizing `identity:get_role(role_id=6b6353b49c6a41b1a157f8997fc52d9a)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:44.258080 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8879f8f-732c-4cae-8393-d2e11bf368a2 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:44.259412 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 552/1113] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:44 2022] GET /identity/v3/roles/6b6353b49c6a41b1a157f8997fc52d9a => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:44.265153 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da80537-4b89-4058-9b55-4bf452f110cd None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:44.359718 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da80537-4b89-4058-9b55-4bf452f110cd None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:44.392083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4874a-80b3-4333-85eb-72e87d07d0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:44.392930 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4874a-80b3-4333-85eb-72e87d07d0f3 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:44.393208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4874a-80b3-4333-85eb-72e87d07d0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:44.393481 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4874a-80b3-4333-85eb-72e87d07d0f3 None admin] PATH_INFO: `/v3/users/ddf3279b7f5d457fb5c7c22b1e4e639e` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:44.406118 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4874a-80b3-4333-85eb-72e87d07d0f3 None admin] RBAC: Authorizing `identity:get_user(user_id=ddf3279b7f5d457fb5c7c22b1e4e639e)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:44.407583 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4874a-80b3-4333-85eb-72e87d07d0f3 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:44.409187 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 562/1114] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:44 2022] GET /identity/v3/users/ddf3279b7f5d457fb5c7c22b1e4e639e => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:44.414697 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8879f8f-732c-4cae-8393-d2e11bf368a2 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:44.430377 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:44.517548 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8879f8f-732c-4cae-8393-d2e11bf368a2 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:44.551987 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28923a0-d1e3-4eb3-b77a-a79701f1f40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:44.552624 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28923a0-d1e3-4eb3-b77a-a79701f1f40f None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:44.552910 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28923a0-d1e3-4eb3-b77a-a79701f1f40f None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:44.553189 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28923a0-d1e3-4eb3-b77a-a79701f1f40f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:44.617867 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28923a0-d1e3-4eb3-b77a-a79701f1f40f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:44.619335 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28923a0-d1e3-4eb3-b77a-a79701f1f40f None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:44.633271 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:44.718001 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 553/1115] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:30:44.724407 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4874a-80b3-4333-85eb-72e87d07d0f3 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:44.733614 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:44.819710 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4874a-80b3-4333-85eb-72e87d07d0f3 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:44.851786 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c223a42-732b-44be-9fec-c291811357bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:44.852625 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c223a42-732b-44be-9fec-c291811357bf None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:44.852902 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c223a42-732b-44be-9fec-c291811357bf None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:44.853170 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c223a42-732b-44be-9fec-c291811357bf None admin] PATH_INFO: `/v3/projects/266b416e205a46aa8c934fbcae0f571d` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:44.854322 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c223a42-732b-44be-9fec-c291811357bf None admin] RBAC: Authorizing `identity:get_project(project_id=266b416e205a46aa8c934fbcae0f571d)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:44.856368 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c223a42-732b-44be-9fec-c291811357bf None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:44.858031 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 563/1116] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:30:44 2022] GET /identity/v3/projects/266b416e205a46aa8c934fbcae0f571d => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:44.864364 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28923a0-d1e3-4eb3-b77a-a79701f1f40f None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:44.962068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28923a0-d1e3-4eb3-b77a-a79701f1f40f None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:44.976007 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:44.995353 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871a18b0-6e63-4e15-82ae-bcbd33449217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:44.995990 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a18b0-6e63-4e15-82ae-bcbd33449217 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:44.996267 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a18b0-6e63-4e15-82ae-bcbd33449217 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:44.996536 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a18b0-6e63-4e15-82ae-bcbd33449217 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:44.997349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871a18b0-6e63-4e15-82ae-bcbd33449217 None admin] RBAC: Adding query filter params (role.id=6b6353b49c6a41b1a157f8997fc52d9a, scope.project.id=266b416e205a46aa8c934fbcae0f571d, user.id=ddf3279b7f5d457fb5c7c22b1e4e639e) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:44.997656 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871a18b0-6e63-4e15-82ae-bcbd33449217 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:44.999089 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871a18b0-6e63-4e15-82ae-bcbd33449217 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:45.004323 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 554/1117] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:44 2022] GET /identity/v3/role_assignments?user.id=ddf3279b7f5d457fb5c7c22b1e4e639e&scope.project.id=266b416e205a46aa8c934fbcae0f571d&role.id=6b6353b49c6a41b1a157f8997fc52d9a => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:47.075614 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffbce8e-c4b9-46d4-9c7c-f941550c2ec3 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:47.076114 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffbce8e-c4b9-46d4-9c7c-f941550c2ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:47.076223 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffbce8e-c4b9-46d4-9c7c-f941550c2ec3 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:47.077541 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 564/1118] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:47.082593 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b4e1d-8a95-4b38-a411-b90be7ca1664 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:47.083023 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b4e1d-8a95-4b38-a411-b90be7ca1664 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:47.083183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b4e1d-8a95-4b38-a411-b90be7ca1664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:47.100131 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-6a3b4e1d-8a95-4b38-a411-b90be7ca1664 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:47.101299 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-6a3b4e1d-8a95-4b38-a411-b90be7ca1664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:47.166185 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 555/1119] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:47.188198 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffbce8e-c4b9-46d4-9c7c-f941550c2ec3 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:47.261410 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:47.287232 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffbce8e-c4b9-46d4-9c7c-f941550c2ec3 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:47.318762 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa217a-89a9-4894-a1f4-4839ff6ac1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:47.319621 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa217a-89a9-4894-a1f4-4839ff6ac1c0 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:47.319901 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa217a-89a9-4894-a1f4-4839ff6ac1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:47.320172 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa217a-89a9-4894-a1f4-4839ff6ac1c0 None admin] PATH_INFO: `/v3/roles/6b6353b49c6a41b1a157f8997fc52d9a` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:47.321613 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa217a-89a9-4894-a1f4-4839ff6ac1c0 None admin] RBAC: Authorizing `identity:get_role(role_id=6b6353b49c6a41b1a157f8997fc52d9a)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:47.323019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa217a-89a9-4894-a1f4-4839ff6ac1c0 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:47.324909 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 565/1120] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:47 2022] GET /identity/v3/roles/6b6353b49c6a41b1a157f8997fc52d9a => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:47.330683 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3b4e1d-8a95-4b38-a411-b90be7ca1664 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:47.425237 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3b4e1d-8a95-4b38-a411-b90be7ca1664 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:47.434577 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:47.459039 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd701db-042c-439e-9eed-3cdb666849be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:47.459889 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd701db-042c-439e-9eed-3cdb666849be None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:47.460173 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd701db-042c-439e-9eed-3cdb666849be None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:47.460446 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd701db-042c-439e-9eed-3cdb666849be None admin] PATH_INFO: `/v3/users/ddf3279b7f5d457fb5c7c22b1e4e639e` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:47.461614 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd701db-042c-439e-9eed-3cdb666849be None admin] RBAC: Authorizing `identity:get_user(user_id=ddf3279b7f5d457fb5c7c22b1e4e639e)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:47.463061 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd701db-042c-439e-9eed-3cdb666849be None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:47.464669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 556/1121] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:47 2022] GET /identity/v3/users/ddf3279b7f5d457fb5c7c22b1e4e639e => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:47.470920 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa217a-89a9-4894-a1f4-4839ff6ac1c0 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:47.566527 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa217a-89a9-4894-a1f4-4839ff6ac1c0 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:47.599621 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224411c0-2ce7-4730-a3f2-cd8c44c57b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:47.600433 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224411c0-2ce7-4730-a3f2-cd8c44c57b2d None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:47.600715 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224411c0-2ce7-4730-a3f2-cd8c44c57b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:47.600988 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224411c0-2ce7-4730-a3f2-cd8c44c57b2d None admin] PATH_INFO: `/v3/projects/266b416e205a46aa8c934fbcae0f571d` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:47.602158 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224411c0-2ce7-4730-a3f2-cd8c44c57b2d None admin] RBAC: Authorizing `identity:get_project(project_id=266b416e205a46aa8c934fbcae0f571d)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:47.603616 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224411c0-2ce7-4730-a3f2-cd8c44c57b2d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:47.605280 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 566/1122] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:30:47 2022] GET /identity/v3/projects/266b416e205a46aa8c934fbcae0f571d => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:47.611033 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd701db-042c-439e-9eed-3cdb666849be None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:47.707919 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd701db-042c-439e-9eed-3cdb666849be None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:47.740528 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b373464-b9f2-42ba-a455-4ec92c2f0eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:47.741452 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b373464-b9f2-42ba-a455-4ec92c2f0eb0 None admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:47.741700 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b373464-b9f2-42ba-a455-4ec92c2f0eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:47.741978 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b373464-b9f2-42ba-a455-4ec92c2f0eb0 None admin] PATH_INFO: `/v3/projects/266b416e205a46aa8c934fbcae0f571d/users/ddf3279b7f5d457fb5c7c22b1e4e639e/roles/6b6353b49c6a41b1a157f8997fc52d9a` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:47.744162 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b373464-b9f2-42ba-a455-4ec92c2f0eb0 None admin] RBAC: Authorizing `identity:create_grant(project_id=266b416e205a46aa8c934fbcae0f571d, user_id=ddf3279b7f5d457fb5c7c22b1e4e639e, role_id=6b6353b49c6a41b1a157f8997fc52d9a)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:47.745592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b373464-b9f2-42ba-a455-4ec92c2f0eb0 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:47.759690 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 557/1123] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Jun 15 14:30:47 2022] PUT /identity/v3/projects/266b416e205a46aa8c934fbcae0f571d/users/ddf3279b7f5d457fb5c7c22b1e4e639e/roles/6b6353b49c6a41b1a157f8997fc52d9a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:30:49.860943 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1349928-cc5d-4763-bc79-c4ab2b63d66e None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:49.861470 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1349928-cc5d-4763-bc79-c4ab2b63d66e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:49.861560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1349928-cc5d-4763-bc79-c4ab2b63d66e None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:49.862883 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 567/1124] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:49.868119 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b97dab-b6a0-4dd5-ab4d-82a96cbe8997 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:49.868476 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b97dab-b6a0-4dd5-ab4d-82a96cbe8997 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:49.868675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b97dab-b6a0-4dd5-ab4d-82a96cbe8997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:49.886187 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-15b97dab-b6a0-4dd5-ab4d-82a96cbe8997 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:49.888009 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-15b97dab-b6a0-4dd5-ab4d-82a96cbe8997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:49.915664 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:49.953574 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 558/1125] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:49.977035 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1349928-cc5d-4763-bc79-c4ab2b63d66e None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:49.989208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:50.078825 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1349928-cc5d-4763-bc79-c4ab2b63d66e None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:50.111177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a38ac-a823-40b3-92af-68a476978d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:50.112027 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a38ac-a823-40b3-92af-68a476978d60 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:50.112314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a38ac-a823-40b3-92af-68a476978d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:50.112592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a38ac-a823-40b3-92af-68a476978d60 None admin] PATH_INFO: `/v3/roles/6b6353b49c6a41b1a157f8997fc52d9a` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:50.114064 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9a38ac-a823-40b3-92af-68a476978d60 None admin] RBAC: Authorizing `identity:get_role(role_id=6b6353b49c6a41b1a157f8997fc52d9a)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:50.115474 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9a38ac-a823-40b3-92af-68a476978d60 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:50.116866 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 568/1126] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:49 2022] GET /identity/v3/roles/6b6353b49c6a41b1a157f8997fc52d9a => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:50.123176 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b97dab-b6a0-4dd5-ab4d-82a96cbe8997 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:50.219116 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b97dab-b6a0-4dd5-ab4d-82a96cbe8997 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:50.252148 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b486276-da9b-4fe4-b1bb-42ad54876973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:50.252939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b486276-da9b-4fe4-b1bb-42ad54876973 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:50.253217 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b486276-da9b-4fe4-b1bb-42ad54876973 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:50.253489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b486276-da9b-4fe4-b1bb-42ad54876973 None admin] PATH_INFO: `/v3/users/ddf3279b7f5d457fb5c7c22b1e4e639e` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:50.254655 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b486276-da9b-4fe4-b1bb-42ad54876973 None admin] RBAC: Authorizing `identity:get_user(user_id=ddf3279b7f5d457fb5c7c22b1e4e639e)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:50.256115 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b486276-da9b-4fe4-b1bb-42ad54876973 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:50.257715 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 559/1127] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:50 2022] GET /identity/v3/users/ddf3279b7f5d457fb5c7c22b1e4e639e => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:50.263297 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a38ac-a823-40b3-92af-68a476978d60 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:50.338218 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:50.359715 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a38ac-a823-40b3-92af-68a476978d60 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:50.392112 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e582d0de-f5b9-47d2-b837-e6a189fde345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:50.392740 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582d0de-f5b9-47d2-b837-e6a189fde345 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:50.393021 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582d0de-f5b9-47d2-b837-e6a189fde345 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:50.393292 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582d0de-f5b9-47d2-b837-e6a189fde345 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:50.457228 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e582d0de-f5b9-47d2-b837-e6a189fde345 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:50.458662 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e582d0de-f5b9-47d2-b837-e6a189fde345 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:50.533650 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:50.562164 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 569/1128] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:30:50.568134 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b486276-da9b-4fe4-b1bb-42ad54876973 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:50.601326 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:50.665622 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b486276-da9b-4fe4-b1bb-42ad54876973 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:50.698185 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d128e439-5b66-487c-8a7f-bd1f84aa9c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:50.699049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128e439-5b66-487c-8a7f-bd1f84aa9c65 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:50.699360 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128e439-5b66-487c-8a7f-bd1f84aa9c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:50.699635 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128e439-5b66-487c-8a7f-bd1f84aa9c65 None admin] PATH_INFO: `/v3/projects/266b416e205a46aa8c934fbcae0f571d` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:50.700785 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d128e439-5b66-487c-8a7f-bd1f84aa9c65 None admin] RBAC: Authorizing `identity:get_project(project_id=266b416e205a46aa8c934fbcae0f571d)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:50.702247 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d128e439-5b66-487c-8a7f-bd1f84aa9c65 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:50.703959 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 560/1129] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:30:50 2022] GET /identity/v3/projects/266b416e205a46aa8c934fbcae0f571d => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:50.710131 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e582d0de-f5b9-47d2-b837-e6a189fde345 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:50.807397 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e582d0de-f5b9-47d2-b837-e6a189fde345 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:50.840897 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0649d-4a9b-467d-ae72-9b650e4adb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:50.841544 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0649d-4a9b-467d-ae72-9b650e4adb6c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:50.841830 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0649d-4a9b-467d-ae72-9b650e4adb6c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:50.842101 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0649d-4a9b-467d-ae72-9b650e4adb6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:50.842919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb0649d-4a9b-467d-ae72-9b650e4adb6c None admin] RBAC: Adding query filter params (role.id=6b6353b49c6a41b1a157f8997fc52d9a, scope.project.id=266b416e205a46aa8c934fbcae0f571d, user.id=ddf3279b7f5d457fb5c7c22b1e4e639e) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:50.843261 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb0649d-4a9b-467d-ae72-9b650e4adb6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:50.844690 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb0649d-4a9b-467d-ae72-9b650e4adb6c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:50.850409 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 570/1130] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:50 2022] GET /identity/v3/role_assignments?user.id=ddf3279b7f5d457fb5c7c22b1e4e639e&scope.project.id=266b416e205a46aa8c934fbcae0f571d&role.id=6b6353b49c6a41b1a157f8997fc52d9a => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:52.986712 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325a52d-8d9e-451b-9379-039d969f1223 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:52.987226 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325a52d-8d9e-451b-9379-039d969f1223 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:52.987341 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325a52d-8d9e-451b-9379-039d969f1223 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:52.988768 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 561/1131] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:52.994070 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e6b25-ea95-4aee-a439-914b70964c69 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:52.994465 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e6b25-ea95-4aee-a439-914b70964c69 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:52.994622 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e6b25-ea95-4aee-a439-914b70964c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:53.011058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-444e6b25-ea95-4aee-a439-914b70964c69 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:53.012250 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-444e6b25-ea95-4aee-a439-914b70964c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:53.047919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:53.082531 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 571/1132] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:30:53.106339 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325a52d-8d9e-451b-9379-039d969f1223 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:53.203773 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325a52d-8d9e-451b-9379-039d969f1223 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:53.205252 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:53.237500 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a0971-4c95-4987-9ace-c4329eddd5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:53.238212 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a0971-4c95-4987-9ace-c4329eddd5d7 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:53.238494 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a0971-4c95-4987-9ace-c4329eddd5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:53.238767 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a0971-4c95-4987-9ace-c4329eddd5d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:53.239952 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a0971-4c95-4987-9ace-c4329eddd5d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:53.241367 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a0971-4c95-4987-9ace-c4329eddd5d7 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:53.242940 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 562/1133] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:53.248615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e6b25-ea95-4aee-a439-914b70964c69 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:53.344500 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e6b25-ea95-4aee-a439-914b70964c69 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:53.376587 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f977f0-d57e-4dbe-9e0b-25d3c376bfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:53.377228 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f977f0-d57e-4dbe-9e0b-25d3c376bfc9 None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:53.377511 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f977f0-d57e-4dbe-9e0b-25d3c376bfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:53.377785 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f977f0-d57e-4dbe-9e0b-25d3c376bfc9 None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:53.378488 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f977f0-d57e-4dbe-9e0b-25d3c376bfc9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:53.379934 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f977f0-d57e-4dbe-9e0b-25d3c376bfc9 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:53.405032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 572/1134] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Jun 15 14:30:53 2022] POST /identity/v3/projects => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:55.523957 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de8ada-e098-48b1-bd19-7e3e7f309726 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:55.524510 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de8ada-e098-48b1-bd19-7e3e7f309726 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:55.524510 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de8ada-e098-48b1-bd19-7e3e7f309726 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:55.525868 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 563/1135] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:55.531269 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9c532-1aa6-474a-81be-13a5e4d608ce None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:55.531681 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9c532-1aa6-474a-81be-13a5e4d608ce None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:55.531834 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9c532-1aa6-474a-81be-13a5e4d608ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:55.548929 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-a3a9c532-1aa6-474a-81be-13a5e4d608ce None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:55.550120 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a3a9c532-1aa6-474a-81be-13a5e4d608ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:55.567565 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:55.616074 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 573/1136] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:30:55.640800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de8ada-e098-48b1-bd19-7e3e7f309726 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:55.739880 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de8ada-e098-48b1-bd19-7e3e7f309726 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:55.773353 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c8f3e-0729-4476-9088-8f5d0e1e3be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:55.774180 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c8f3e-0729-4476-9088-8f5d0e1e3be6 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:55.774442 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c8f3e-0729-4476-9088-8f5d0e1e3be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:55.774719 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c8f3e-0729-4476-9088-8f5d0e1e3be6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:55.775907 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395c8f3e-0729-4476-9088-8f5d0e1e3be6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:55.777340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395c8f3e-0729-4476-9088-8f5d0e1e3be6 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:55.779534 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 564/1137] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:55.785131 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9c532-1aa6-474a-81be-13a5e4d608ce None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:55.880007 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9c532-1aa6-474a-81be-13a5e4d608ce None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:55.912697 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d73c4cb-2287-4793-a877-23ee00ed880d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:55.913349 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73c4cb-2287-4793-a877-23ee00ed880d None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:55.913631 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73c4cb-2287-4793-a877-23ee00ed880d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:55.913905 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73c4cb-2287-4793-a877-23ee00ed880d None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:55.914611 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d73c4cb-2287-4793-a877-23ee00ed880d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:55.916071 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d73c4cb-2287-4793-a877-23ee00ed880d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:55.935343 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 574/1138] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Jun 15 14:30:55 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:58.072936 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91949f72-c1da-4099-9aa2-cc1992ff453c None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:58.073478 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91949f72-c1da-4099-9aa2-cc1992ff453c None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:58.073478 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91949f72-c1da-4099-9aa2-cc1992ff453c None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:58.075431 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 565/1139] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:30:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:58.080763 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24e082-7967-400d-82ee-66b24e83b489 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:58.081191 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24e082-7967-400d-82ee-66b24e83b489 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:58.081323 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24e082-7967-400d-82ee-66b24e83b489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:58.098513 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-bd24e082-7967-400d-82ee-66b24e83b489 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:58.099775 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-bd24e082-7967-400d-82ee-66b24e83b489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:58.102863 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:58.166674 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 575/1140] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:58.190657 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91949f72-c1da-4099-9aa2-cc1992ff453c None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:58.230582 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:58.289639 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91949f72-c1da-4099-9aa2-cc1992ff453c None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:58.322190 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd730c-2597-4040-9fc2-3764abec46ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:58.323023 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd730c-2597-4040-9fc2-3764abec46ea None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:58.323343 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd730c-2597-4040-9fc2-3764abec46ea None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:58.323623 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd730c-2597-4040-9fc2-3764abec46ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:58.333344 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbd730c-2597-4040-9fc2-3764abec46ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:58.341543 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-1fbd730c-2597-4040-9fc2-3764abec46ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:58.341543 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:58.341543 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:58.341543 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:58.341543 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:58.341543 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:58.341543 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:58.341543 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:58.341543 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:58.341543 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:58.341543 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:58.341543 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:58.341543 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:58.341543 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:58.341543 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:58.341543 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:58.342630 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:58.342630 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:58.342630 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:58.342630 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:58.342630 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:58.342630 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:58.342630 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:58.342630 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:58.342630 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:58.342630 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:58.342630 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:58.342630 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:58.342630 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:58.342630 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:58.342630 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:58.342630 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:58.343596 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:58.343596 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:58.343596 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:58.343596 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:58.343596 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:58.343596 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:58.343596 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:58.343596 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:58.343596 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:58.343596 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:58.343596 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:58.343596 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:30:58.343596 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 566/1141] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:30:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:58.348697 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24e082-7967-400d-82ee-66b24e83b489 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:58.447781 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24e082-7967-400d-82ee-66b24e83b489 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:58.481115 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c8f311-3c63-4c9d-b464-84e3609fc0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:58.481773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c8f311-3c63-4c9d-b464-84e3609fc0c8 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:58.482054 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c8f311-3c63-4c9d-b464-84e3609fc0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:58.482329 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c8f311-3c63-4c9d-b464-84e3609fc0c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:58.483092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c8f311-3c63-4c9d-b464-84e3609fc0c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:58.483400 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c8f311-3c63-4c9d-b464-84e3609fc0c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:58.484770 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c8f311-3c63-4c9d-b464-84e3609fc0c8 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:58.492596 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 576/1142] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:58.499212 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd730c-2597-4040-9fc2-3764abec46ea None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:58.592604 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:58.596629 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd730c-2597-4040-9fc2-3764abec46ea None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:58.629938 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c134fdf5-748c-4c96-b8b0-05662840e392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:58.630795 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134fdf5-748c-4c96-b8b0-05662840e392 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:58.631109 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134fdf5-748c-4c96-b8b0-05662840e392 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:58.631389 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134fdf5-748c-4c96-b8b0-05662840e392 None admin] PATH_INFO: `/v3/users/888872d2dcfe4778a3efa9376669bd07` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:58.644196 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c134fdf5-748c-4c96-b8b0-05662840e392 None admin] RBAC: Authorizing `identity:get_user(user_id=888872d2dcfe4778a3efa9376669bd07)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:58.645631 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c134fdf5-748c-4c96-b8b0-05662840e392 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:58.647552 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 567/1143] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:58 2022] GET /identity/v3/users/888872d2dcfe4778a3efa9376669bd07 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:58.653278 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c8f311-3c63-4c9d-b464-84e3609fc0c8 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:58.656119 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:58.752235 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c8f311-3c63-4c9d-b464-84e3609fc0c8 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:58.784726 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7750944-5dce-43da-982e-3a8bb882c8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:58.785373 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7750944-5dce-43da-982e-3a8bb882c8c2 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:58.785658 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7750944-5dce-43da-982e-3a8bb882c8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:58.785935 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7750944-5dce-43da-982e-3a8bb882c8c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:58.850046 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7750944-5dce-43da-982e-3a8bb882c8c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:58.851467 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7750944-5dce-43da-982e-3a8bb882c8c2 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:58.853217 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:58.946676 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 577/1144] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:30:58.952538 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c134fdf5-748c-4c96-b8b0-05662840e392 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:59.049555 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c134fdf5-748c-4c96-b8b0-05662840e392 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:59.091270 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059b4542-d09c-4a29-81ab-fffec887ee13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:59.092199 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b4542-d09c-4a29-81ab-fffec887ee13 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:59.092508 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b4542-d09c-4a29-81ab-fffec887ee13 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:59.092795 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b4542-d09c-4a29-81ab-fffec887ee13 None admin] PATH_INFO: `/v3/projects/f9b21410bf3e4675a3a37a8f55c11e58` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:59.094009 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059b4542-d09c-4a29-81ab-fffec887ee13 None admin] RBAC: Authorizing `identity:get_project(project_id=f9b21410bf3e4675a3a37a8f55c11e58)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:59.095481 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059b4542-d09c-4a29-81ab-fffec887ee13 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:59.097194 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 568/1145] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:30:58 2022] GET /identity/v3/projects/f9b21410bf3e4675a3a37a8f55c11e58 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:59.103278 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7750944-5dce-43da-982e-3a8bb882c8c2 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:59.198562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7750944-5dce-43da-982e-3a8bb882c8c2 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:59.200577 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:59.233045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6802081-43e3-452c-afb9-d638f93838b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:59.233680 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6802081-43e3-452c-afb9-d638f93838b2 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:59.233960 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6802081-43e3-452c-afb9-d638f93838b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:59.234231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6802081-43e3-452c-afb9-d638f93838b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:59.235076 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6802081-43e3-452c-afb9-d638f93838b2 None admin] RBAC: Adding query filter params (role.id=ed1cfa2fcf724a4d84f5acea100930c7, scope.project.id=f9b21410bf3e4675a3a37a8f55c11e58, user.id=888872d2dcfe4778a3efa9376669bd07) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:59.235388 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6802081-43e3-452c-afb9-d638f93838b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:59.236805 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6802081-43e3-452c-afb9-d638f93838b2 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:59.242083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 578/1146] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:59 2022] GET /identity/v3/role_assignments?user.id=888872d2dcfe4778a3efa9376669bd07&scope.project.id=f9b21410bf3e4675a3a37a8f55c11e58&role.id=ed1cfa2fcf724a4d84f5acea100930c7 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:01.333104 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39cfac-ee6b-4889-b7ce-df70604f321d None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:01.333627 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39cfac-ee6b-4889-b7ce-df70604f321d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:01.333737 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39cfac-ee6b-4889-b7ce-df70604f321d None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:01.335067 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 569/1147] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:31:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:01.340647 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141d8ed-c900-4e48-bd39-924b3031fa6c None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:01.341049 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141d8ed-c900-4e48-bd39-924b3031fa6c None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:01.341192 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141d8ed-c900-4e48-bd39-924b3031fa6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:01.358166 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-e141d8ed-c900-4e48-bd39-924b3031fa6c None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:01.359358 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e141d8ed-c900-4e48-bd39-924b3031fa6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:01.424059 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 579/1148] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:31:01.447109 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39cfac-ee6b-4889-b7ce-df70604f321d None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:01.472739 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:01.547103 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39cfac-ee6b-4889-b7ce-df70604f321d None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:01.579635 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d912d54f-53df-4ac2-ae5a-c86781b0d664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:01.580409 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912d54f-53df-4ac2-ae5a-c86781b0d664 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:01.580690 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912d54f-53df-4ac2-ae5a-c86781b0d664 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:01.580962 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912d54f-53df-4ac2-ae5a-c86781b0d664 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:01.593070 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d912d54f-53df-4ac2-ae5a-c86781b0d664 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:01.600559 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-d912d54f-53df-4ac2-ae5a-c86781b0d664 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:01.600559 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:01.600559 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:01.600559 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:01.600559 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:01.600559 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:01.600559 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:01.600559 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:01.600559 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:01.600559 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:01.600559 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:01.600559 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:01.600559 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:01.600559 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:01.600559 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:01.600559 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:01.601618 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:01.601618 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:01.601618 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:01.601618 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:01.601618 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:01.601618 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:01.601618 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:01.601618 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:01.601618 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:01.601618 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:01.601618 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:01.601618 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:01.601618 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:01.601618 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:01.601618 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:01.601618 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:01.602607 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:01.602607 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:01.602607 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:01.602607 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:01.602607 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:01.602607 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:01.602607 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:01.602607 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:01.602607 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:01.602607 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:01.602607 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:01.602607 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:31:01.602607 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 570/1149] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:31:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:01.607904 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e141d8ed-c900-4e48-bd39-924b3031fa6c None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:01.698488 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:01.705142 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e141d8ed-c900-4e48-bd39-924b3031fa6c None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:01.738442 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8abb442-3e9d-48bd-a125-34c4bcdd5840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:01.739134 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8abb442-3e9d-48bd-a125-34c4bcdd5840 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:01.739422 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8abb442-3e9d-48bd-a125-34c4bcdd5840 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:01.739697 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8abb442-3e9d-48bd-a125-34c4bcdd5840 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:01.740441 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8abb442-3e9d-48bd-a125-34c4bcdd5840 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:01.740750 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8abb442-3e9d-48bd-a125-34c4bcdd5840 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:01.742112 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8abb442-3e9d-48bd-a125-34c4bcdd5840 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:01.749930 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 580/1150] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:01.755625 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d912d54f-53df-4ac2-ae5a-c86781b0d664 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:01.838830 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:01.854480 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d912d54f-53df-4ac2-ae5a-c86781b0d664 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:01.887587 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a506add-b9c1-4689-9143-8e45c9ddbd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:01.888444 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a506add-b9c1-4689-9143-8e45c9ddbd15 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:01.888730 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a506add-b9c1-4689-9143-8e45c9ddbd15 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:01.889008 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a506add-b9c1-4689-9143-8e45c9ddbd15 None admin] PATH_INFO: `/v3/users/888872d2dcfe4778a3efa9376669bd07` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:01.890189 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a506add-b9c1-4689-9143-8e45c9ddbd15 None admin] RBAC: Authorizing `identity:get_user(user_id=888872d2dcfe4778a3efa9376669bd07)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:01.891655 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a506add-b9c1-4689-9143-8e45c9ddbd15 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:01.893266 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 571/1151] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:31:01 2022] GET /identity/v3/users/888872d2dcfe4778a3efa9376669bd07 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:01.899206 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8abb442-3e9d-48bd-a125-34c4bcdd5840 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:01.994511 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8abb442-3e9d-48bd-a125-34c4bcdd5840 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:02.027671 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0096ad-1b0f-4837-88d5-8ca5b7caf915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:02.028514 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0096ad-1b0f-4837-88d5-8ca5b7caf915 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:02.028799 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0096ad-1b0f-4837-88d5-8ca5b7caf915 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:02.029077 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0096ad-1b0f-4837-88d5-8ca5b7caf915 None admin] PATH_INFO: `/v3/projects/f9b21410bf3e4675a3a37a8f55c11e58` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:02.030250 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0096ad-1b0f-4837-88d5-8ca5b7caf915 None admin] RBAC: Authorizing `identity:get_project(project_id=f9b21410bf3e4675a3a37a8f55c11e58)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:02.031721 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0096ad-1b0f-4837-88d5-8ca5b7caf915 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:02.033408 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 581/1152] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:31:01 2022] GET /identity/v3/projects/f9b21410bf3e4675a3a37a8f55c11e58 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:02.039062 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a506add-b9c1-4689-9143-8e45c9ddbd15 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:02.135532 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a506add-b9c1-4689-9143-8e45c9ddbd15 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:02.168656 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d7128-22e0-4e1f-8eeb-a59a7df19c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:02.169527 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d7128-22e0-4e1f-8eeb-a59a7df19c0f None admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:02.169815 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d7128-22e0-4e1f-8eeb-a59a7df19c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:02.170093 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d7128-22e0-4e1f-8eeb-a59a7df19c0f None admin] PATH_INFO: `/v3/projects/f9b21410bf3e4675a3a37a8f55c11e58/users/888872d2dcfe4778a3efa9376669bd07/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:02.172064 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d7128-22e0-4e1f-8eeb-a59a7df19c0f None admin] RBAC: Authorizing `identity:create_grant(project_id=f9b21410bf3e4675a3a37a8f55c11e58, user_id=888872d2dcfe4778a3efa9376669bd07, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:02.173482 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d7128-22e0-4e1f-8eeb-a59a7df19c0f None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:02.188949 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 572/1153] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Jun 15 14:31:02 2022] PUT /identity/v3/projects/f9b21410bf3e4675a3a37a8f55c11e58/users/888872d2dcfe4778a3efa9376669bd07/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:31:04.289985 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd66095-adba-42e2-ab74-b76096a2c0f1 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:04.290503 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd66095-adba-42e2-ab74-b76096a2c0f1 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:04.290618 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd66095-adba-42e2-ab74-b76096a2c0f1 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:04.291966 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 582/1154] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:31:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:04.297813 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb34d9-b1a6-4e5e-86dc-be93da9ed366 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:04.298175 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb34d9-b1a6-4e5e-86dc-be93da9ed366 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:04.298369 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb34d9-b1a6-4e5e-86dc-be93da9ed366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:04.314580 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-ebeb34d9-b1a6-4e5e-86dc-be93da9ed366 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:04.315769 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ebeb34d9-b1a6-4e5e-86dc-be93da9ed366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:04.322797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:04.381314 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 573/1155] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:31:04.405528 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd66095-adba-42e2-ab74-b76096a2c0f1 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:04.432107 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:04.504838 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd66095-adba-42e2-ab74-b76096a2c0f1 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:04.537590 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9edcd8-b113-4a7c-b79d-5c813f189212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:04.538366 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9edcd8-b113-4a7c-b79d-5c813f189212 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:04.538646 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9edcd8-b113-4a7c-b79d-5c813f189212 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:04.538916 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9edcd8-b113-4a7c-b79d-5c813f189212 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:04.549136 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9edcd8-b113-4a7c-b79d-5c813f189212 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:04.556606 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-0a9edcd8-b113-4a7c-b79d-5c813f189212 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:04.556606 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:04.556606 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:04.556606 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:04.556606 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:04.556606 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:04.556606 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:04.556606 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:04.556606 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:04.556606 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:04.556606 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:04.556606 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:04.556606 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:04.556606 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:04.556606 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:04.556606 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:04.557667 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:04.557667 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:04.557667 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:04.557667 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:04.557667 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:04.557667 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:04.557667 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:04.557667 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:04.557667 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:04.557667 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:04.557667 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:04.557667 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:04.557667 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:04.557667 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:04.557667 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:04.557667 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:04.558861 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:04.558861 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:04.558861 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:04.558861 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:04.558861 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:04.558861 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:04.558861 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:04.558861 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:04.558861 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:04.558861 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:04.558861 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:04.558861 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:31:04.558861 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 583/1156] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:31:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:04.563831 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb34d9-b1a6-4e5e-86dc-be93da9ed366 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:04.659104 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb34d9-b1a6-4e5e-86dc-be93da9ed366 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:04.691158 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd60abd-a57f-47a5-8f85-eb6814f756c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:04.691808 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd60abd-a57f-47a5-8f85-eb6814f756c1 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:04.692093 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd60abd-a57f-47a5-8f85-eb6814f756c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:04.692371 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd60abd-a57f-47a5-8f85-eb6814f756c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:04.693110 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd60abd-a57f-47a5-8f85-eb6814f756c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:04.693416 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd60abd-a57f-47a5-8f85-eb6814f756c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:04.694778 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd60abd-a57f-47a5-8f85-eb6814f756c1 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:04.703415 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 574/1157] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:04.708993 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9edcd8-b113-4a7c-b79d-5c813f189212 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:04.791304 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:04.805817 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9edcd8-b113-4a7c-b79d-5c813f189212 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:04.843041 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb6305-0e4e-4a8b-9caf-5ec314413c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:04.843910 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb6305-0e4e-4a8b-9caf-5ec314413c9a None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:04.844194 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb6305-0e4e-4a8b-9caf-5ec314413c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:04.844471 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb6305-0e4e-4a8b-9caf-5ec314413c9a None admin] PATH_INFO: `/v3/users/888872d2dcfe4778a3efa9376669bd07` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:04.845659 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cb6305-0e4e-4a8b-9caf-5ec314413c9a None admin] RBAC: Authorizing `identity:get_user(user_id=888872d2dcfe4778a3efa9376669bd07)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:04.847120 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cb6305-0e4e-4a8b-9caf-5ec314413c9a None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:04.848770 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 584/1158] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:31:04 2022] GET /identity/v3/users/888872d2dcfe4778a3efa9376669bd07 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:04.854639 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd60abd-a57f-47a5-8f85-eb6814f756c1 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:04.858345 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:04.952109 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd60abd-a57f-47a5-8f85-eb6814f756c1 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:04.983705 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b617b-7348-4f98-a999-6f0a1a3d6337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:04.984370 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b617b-7348-4f98-a999-6f0a1a3d6337 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:04.984653 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b617b-7348-4f98-a999-6f0a1a3d6337 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:04.984928 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b617b-7348-4f98-a999-6f0a1a3d6337 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:05.049180 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894b617b-7348-4f98-a999-6f0a1a3d6337 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:05.050602 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894b617b-7348-4f98-a999-6f0a1a3d6337 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:05.053134 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:05.154397 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 575/1159] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:31:05.160157 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb6305-0e4e-4a8b-9caf-5ec314413c9a None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:05.257301 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb6305-0e4e-4a8b-9caf-5ec314413c9a None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:05.289662 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86954770-fdaf-4e47-b297-fd183e839a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:05.290498 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86954770-fdaf-4e47-b297-fd183e839a4b None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:05.291052 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86954770-fdaf-4e47-b297-fd183e839a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:05.291052 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86954770-fdaf-4e47-b297-fd183e839a4b None admin] PATH_INFO: `/v3/projects/f9b21410bf3e4675a3a37a8f55c11e58` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:05.292214 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86954770-fdaf-4e47-b297-fd183e839a4b None admin] RBAC: Authorizing `identity:get_project(project_id=f9b21410bf3e4675a3a37a8f55c11e58)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:05.294188 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86954770-fdaf-4e47-b297-fd183e839a4b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:05.295891 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 585/1160] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:31:05 2022] GET /identity/v3/projects/f9b21410bf3e4675a3a37a8f55c11e58 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:05.301969 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b617b-7348-4f98-a999-6f0a1a3d6337 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:05.397419 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b617b-7348-4f98-a999-6f0a1a3d6337 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:05.399937 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:05.430839 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e6804-6f9b-4a9c-9334-1d7cb5ea6c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:05.431453 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e6804-6f9b-4a9c-9334-1d7cb5ea6c25 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:05.431739 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e6804-6f9b-4a9c-9334-1d7cb5ea6c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:05.432015 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e6804-6f9b-4a9c-9334-1d7cb5ea6c25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:05.432831 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e6804-6f9b-4a9c-9334-1d7cb5ea6c25 None admin] RBAC: Adding query filter params (role.id=ed1cfa2fcf724a4d84f5acea100930c7, scope.project.id=f9b21410bf3e4675a3a37a8f55c11e58, user.id=888872d2dcfe4778a3efa9376669bd07) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:05.433143 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e6804-6f9b-4a9c-9334-1d7cb5ea6c25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:05.435138 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e6804-6f9b-4a9c-9334-1d7cb5ea6c25 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:05.441197 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 576/1161] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:31:05 2022] GET /identity/v3/role_assignments?user.id=888872d2dcfe4778a3efa9376669bd07&scope.project.id=f9b21410bf3e4675a3a37a8f55c11e58&role.id=ed1cfa2fcf724a4d84f5acea100930c7 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:07.564689 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c44dc-e0ae-40a5-bac6-207c8ebb8dc0 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:07.565235 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c44dc-e0ae-40a5-bac6-207c8ebb8dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:07.565326 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c44dc-e0ae-40a5-bac6-207c8ebb8dc0 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:07.566652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 586/1162] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:07.571729 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40bd6d-2c83-4c1b-afed-535978a8a92d None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:07.572137 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40bd6d-2c83-4c1b-afed-535978a8a92d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:07.572291 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40bd6d-2c83-4c1b-afed-535978a8a92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:07.589367 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-ef40bd6d-2c83-4c1b-afed-535978a8a92d None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:07.591201 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ef40bd6d-2c83-4c1b-afed-535978a8a92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:07.656315 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 577/1163] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:31:07.679230 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c44dc-e0ae-40a5-bac6-207c8ebb8dc0 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:07.703628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:07.781822 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c44dc-e0ae-40a5-bac6-207c8ebb8dc0 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:07.814614 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aef4e3-e9ee-4c11-9d75-2de607c3ee60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:07.815335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aef4e3-e9ee-4c11-9d75-2de607c3ee60 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:07.815567 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aef4e3-e9ee-4c11-9d75-2de607c3ee60 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:07.815844 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aef4e3-e9ee-4c11-9d75-2de607c3ee60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:07.880859 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aef4e3-e9ee-4c11-9d75-2de607c3ee60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:07.882297 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aef4e3-e9ee-4c11-9d75-2de607c3ee60 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:07.905499 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:07.976867 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 587/1164] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:31:07.982812 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef40bd6d-2c83-4c1b-afed-535978a8a92d None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:08.076261 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:08.080399 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef40bd6d-2c83-4c1b-afed-535978a8a92d None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:08.112403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0621b936-5ad2-4610-a68f-f01dcf676bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:08.113179 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621b936-5ad2-4610-a68f-f01dcf676bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:08.113458 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621b936-5ad2-4610-a68f-f01dcf676bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:08.113728 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621b936-5ad2-4610-a68f-f01dcf676bc8 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:08.120152 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0621b936-5ad2-4610-a68f-f01dcf676bc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:08.121574 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0621b936-5ad2-4610-a68f-f01dcf676bc8 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:08.135787 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-0621b936-5ad2-4610-a68f-f01dcf676bc8 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 15 14:31:08.135787 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:08.135787 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 15 14:31:08.135787 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 15 14:31:08.135787 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:31:08.135787 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:31:08.135787 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:31:08.135787 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:31:08.135787 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 15 14:31:08.135787 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:31:08.135787 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:31:08.135787 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:31:08.135787 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:08.135787 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:08.135787 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:08.135787 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:08.135787 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:08.136943 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:08.136943 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:08.136943 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:08.136943 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:08.136943 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:08.136943 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:08.136943 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:08.136943 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:08.136943 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:08.136943 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:08.136943 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:08.136943 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:08.136943 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:08.136943 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:08.136943 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:08.136943 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:08.138774 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:08.138774 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:08.138774 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:08.138774 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:08.138774 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:08.138774 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:08.138774 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:08.138774 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:08.138774 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:08.138774 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:08.138774 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 15 14:31:08.138774 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:08.138774 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 15 14:31:08.138774 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:31:08.138774 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 578/1165] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Jun 15 14:31:07 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:08.143427 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aef4e3-e9ee-4c11-9d75-2de607c3ee60 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:08.237730 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aef4e3-e9ee-4c11-9d75-2de607c3ee60 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:08.259743 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:08.271191 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499f0f1c-0258-4d89-b086-353c194928e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:08.271840 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f0f1c-0258-4d89-b086-353c194928e8 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:08.272123 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f0f1c-0258-4d89-b086-353c194928e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:08.272397 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f0f1c-0258-4d89-b086-353c194928e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:08.273365 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499f0f1c-0258-4d89-b086-353c194928e8 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:08.273669 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499f0f1c-0258-4d89-b086-353c194928e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:08.275053 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499f0f1c-0258-4d89-b086-353c194928e8 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:08.282139 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 588/1166] 10.222.0.45 () {56 vars in 1115 bytes} [Wed Jun 15 14:31:08 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:10.362168 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531dd95e-8450-4160-9eda-afe29d4cc6f8 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:10.362705 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531dd95e-8450-4160-9eda-afe29d4cc6f8 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:10.362705 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531dd95e-8450-4160-9eda-afe29d4cc6f8 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:10.364132 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 579/1167] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:10.369264 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a842ed-9387-4927-9bfd-f80524c16845 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:10.369623 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a842ed-9387-4927-9bfd-f80524c16845 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:10.369829 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a842ed-9387-4927-9bfd-f80524c16845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:10.386746 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-d8a842ed-9387-4927-9bfd-f80524c16845 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:10.388438 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d8a842ed-9387-4927-9bfd-f80524c16845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:10.460806 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 589/1168] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:31:10.484146 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531dd95e-8450-4160-9eda-afe29d4cc6f8 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:10.584384 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531dd95e-8450-4160-9eda-afe29d4cc6f8 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:10.617307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b65ce-1b76-45b3-bd7e-7b7147bcbfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:10.617897 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b65ce-1b76-45b3-bd7e-7b7147bcbfb0 None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:10.618182 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b65ce-1b76-45b3-bd7e-7b7147bcbfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:10.618462 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b65ce-1b76-45b3-bd7e-7b7147bcbfb0 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:10.619219 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6b65ce-1b76-45b3-bd7e-7b7147bcbfb0 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:10.620582 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6b65ce-1b76-45b3-bd7e-7b7147bcbfb0 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:10.635998 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 580/1169] 10.222.0.45 () {60 vars in 1138 bytes} [Wed Jun 15 14:31:10 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:31:12.761073 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392964c3-07f2-4f34-9475-c97cb2a5d057 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:12.761494 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392964c3-07f2-4f34-9475-c97cb2a5d057 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:12.761620 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392964c3-07f2-4f34-9475-c97cb2a5d057 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:12.763016 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 590/1170] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:12.768102 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118b14d-c6fd-4cd7-9a28-2955f39c5f12 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:12.768539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118b14d-c6fd-4cd7-9a28-2955f39c5f12 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:12.768649 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118b14d-c6fd-4cd7-9a28-2955f39c5f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:12.786127 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-5118b14d-c6fd-4cd7-9a28-2955f39c5f12 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:12.787336 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-5118b14d-c6fd-4cd7-9a28-2955f39c5f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:12.800622 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:12.852745 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 581/1171] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:31:12.876648 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392964c3-07f2-4f34-9475-c97cb2a5d057 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:12.974309 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392964c3-07f2-4f34-9475-c97cb2a5d057 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:12.990620 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:13.008568 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2337d6e-cda2-46d1-a37c-141c1e9abd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:13.009418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2337d6e-cda2-46d1-a37c-141c1e9abd82 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:13.009701 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2337d6e-cda2-46d1-a37c-141c1e9abd82 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:13.009975 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2337d6e-cda2-46d1-a37c-141c1e9abd82 None admin] PATH_INFO: `/v3/domains/6e5948847e7148c1b188fc7e5e0e1bfc` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:13.020085 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2337d6e-cda2-46d1-a37c-141c1e9abd82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=6e5948847e7148c1b188fc7e5e0e1bfc)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:13.021534 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2337d6e-cda2-46d1-a37c-141c1e9abd82 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:13.023541 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 591/1172] 10.222.0.45 () {56 vars in 1183 bytes} [Wed Jun 15 14:31:12 2022] GET /identity/v3/domains/6e5948847e7148c1b188fc7e5e0e1bfc => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:13.029766 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118b14d-c6fd-4cd7-9a28-2955f39c5f12 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:13.129982 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118b14d-c6fd-4cd7-9a28-2955f39c5f12 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:13.166425 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f6863-6721-4ca9-a50c-b6f5f94b0950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:13.167102 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f6863-6721-4ca9-a50c-b6f5f94b0950 None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:13.167387 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f6863-6721-4ca9-a50c-b6f5f94b0950 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:13.167664 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f6863-6721-4ca9-a50c-b6f5f94b0950 None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:13.168378 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44f6863-6721-4ca9-a50c-b6f5f94b0950 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:13.170382 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44f6863-6721-4ca9-a50c-b6f5f94b0950 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:13.188728 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:13.195456 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 582/1173] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Jun 15 14:31:13 2022] POST /identity/v3/projects => generated 361 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:31:15.342190 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00847e9-c10f-4522-a60a-4d97af6db1bc None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:15.342729 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00847e9-c10f-4522-a60a-4d97af6db1bc None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:15.342844 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00847e9-c10f-4522-a60a-4d97af6db1bc None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:15.344262 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 592/1174] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:15.349337 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9c4c0-dcba-4ce3-8134-6a51fefb1b24 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:15.349713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9c4c0-dcba-4ce3-8134-6a51fefb1b24 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:15.349906 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9c4c0-dcba-4ce3-8134-6a51fefb1b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:15.366389 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-0df9c4c0-dcba-4ce3-8134-6a51fefb1b24 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:15.367592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-0df9c4c0-dcba-4ce3-8134-6a51fefb1b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:15.432486 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 583/1175] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:31:15.457698 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00847e9-c10f-4522-a60a-4d97af6db1bc None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:15.555095 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00847e9-c10f-4522-a60a-4d97af6db1bc None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:15.587303 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7309d78-df24-40e3-a868-a43ccac53cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:15.588145 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7309d78-df24-40e3-a868-a43ccac53cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:15.588426 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7309d78-df24-40e3-a868-a43ccac53cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:15.588699 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7309d78-df24-40e3-a868-a43ccac53cb0 None admin] PATH_INFO: `/v3/domains/6e5948847e7148c1b188fc7e5e0e1bfc` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:15.589858 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7309d78-df24-40e3-a868-a43ccac53cb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=6e5948847e7148c1b188fc7e5e0e1bfc)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:15.591333 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7309d78-df24-40e3-a868-a43ccac53cb0 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:15.592985 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 593/1176] 10.222.0.45 () {56 vars in 1183 bytes} [Wed Jun 15 14:31:15 2022] GET /identity/v3/domains/6e5948847e7148c1b188fc7e5e0e1bfc => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:15.598509 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df9c4c0-dcba-4ce3-8134-6a51fefb1b24 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:15.695577 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df9c4c0-dcba-4ce3-8134-6a51fefb1b24 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:15.723702 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:15.729710 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c849b0e8-1a7f-423c-8326-87afb1d5abaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:15.730376 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849b0e8-1a7f-423c-8326-87afb1d5abaa None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:15.730655 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849b0e8-1a7f-423c-8326-87afb1d5abaa None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:15.730928 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849b0e8-1a7f-423c-8326-87afb1d5abaa None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:15.732173 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c849b0e8-1a7f-423c-8326-87afb1d5abaa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:15.733594 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c849b0e8-1a7f-423c-8326-87afb1d5abaa None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:15.752934 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 584/1177] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Jun 15 14:31:15 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:31:17.892745 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54199ce-9a53-47c4-b637-766a0fe5269b None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:17.893271 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54199ce-9a53-47c4-b637-766a0fe5269b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:17.893361 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54199ce-9a53-47c4-b637-766a0fe5269b None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:17.894688 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 594/1178] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:31:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:17.900199 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8ce92-a41d-47b9-9ff4-da035b0bc88d None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:17.900603 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8ce92-a41d-47b9-9ff4-da035b0bc88d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:17.900748 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8ce92-a41d-47b9-9ff4-da035b0bc88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:17.917613 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-79e8ce92-a41d-47b9-9ff4-da035b0bc88d None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:17.918773 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-79e8ce92-a41d-47b9-9ff4-da035b0bc88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:17.982409 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 585/1179] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:31:18.005614 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54199ce-9a53-47c4-b637-766a0fe5269b None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:18.052692 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:18.106859 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54199ce-9a53-47c4-b637-766a0fe5269b None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:18.139297 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51ee34-ff23-4552-bb21-0fe3987ed5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:18.140072 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51ee34-ff23-4552-bb21-0fe3987ed5b1 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:18.140354 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51ee34-ff23-4552-bb21-0fe3987ed5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:18.140635 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51ee34-ff23-4552-bb21-0fe3987ed5b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:18.151771 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51ee34-ff23-4552-bb21-0fe3987ed5b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:18.159887 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-8c51ee34-ff23-4552-bb21-0fe3987ed5b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:18.159887 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:18.159887 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:18.159887 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:18.159887 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:18.159887 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:18.159887 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:18.159887 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:18.159887 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:18.159887 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:18.159887 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:18.159887 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:18.159887 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:18.159887 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:18.159887 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:18.159887 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:18.161044 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:18.161044 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:18.161044 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:18.161044 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:18.161044 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:18.161044 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:18.161044 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:18.161044 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:18.161044 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:18.161044 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:18.161044 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:18.161044 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:18.161044 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:18.161044 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:18.161044 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:18.161044 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:18.162119 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:18.162119 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:18.162119 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:18.162119 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:18.162119 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:18.162119 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:18.162119 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:18.162119 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:18.162119 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:18.162119 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:18.162119 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:18.162119 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:31:18.162119 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 595/1180] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:31:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:18.167213 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e8ce92-a41d-47b9-9ff4-da035b0bc88d None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:18.261705 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e8ce92-a41d-47b9-9ff4-da035b0bc88d None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:18.277909 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:18.296365 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e0e6cf-fe0b-400c-8f02-b0108932adf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:18.297014 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0e6cf-fe0b-400c-8f02-b0108932adf7 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:18.297295 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0e6cf-fe0b-400c-8f02-b0108932adf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:18.297571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0e6cf-fe0b-400c-8f02-b0108932adf7 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:18.298303 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e0e6cf-fe0b-400c-8f02-b0108932adf7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:18.298608 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e0e6cf-fe0b-400c-8f02-b0108932adf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:18.299998 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e0e6cf-fe0b-400c-8f02-b0108932adf7 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:18.307746 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 586/1181] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:18.314202 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51ee34-ff23-4552-bb21-0fe3987ed5b1 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:18.410143 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51ee34-ff23-4552-bb21-0fe3987ed5b1 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:18.412410 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:18.443113 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0989df7-8274-46fb-b787-c9e44d24e022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:18.443969 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0989df7-8274-46fb-b787-c9e44d24e022 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:18.444249 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0989df7-8274-46fb-b787-c9e44d24e022 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:18.444522 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0989df7-8274-46fb-b787-c9e44d24e022 None admin] PATH_INFO: `/v3/users/95c673ae8c444ced91db11beeb64fb94` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:18.457727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0989df7-8274-46fb-b787-c9e44d24e022 None admin] RBAC: Authorizing `identity:get_user(user_id=95c673ae8c444ced91db11beeb64fb94)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:18.459183 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0989df7-8274-46fb-b787-c9e44d24e022 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:18.460830 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 596/1182] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:31:18 2022] GET /identity/v3/users/95c673ae8c444ced91db11beeb64fb94 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:18.467208 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e0e6cf-fe0b-400c-8f02-b0108932adf7 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:18.563907 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e0e6cf-fe0b-400c-8f02-b0108932adf7 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:18.596302 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b975c5-ed7b-431f-83d7-9faed1a96936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:18.596933 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b975c5-ed7b-431f-83d7-9faed1a96936 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:18.597219 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b975c5-ed7b-431f-83d7-9faed1a96936 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:18.597497 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b975c5-ed7b-431f-83d7-9faed1a96936 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:18.641709 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:18.662886 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b975c5-ed7b-431f-83d7-9faed1a96936 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:18.664362 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b975c5-ed7b-431f-83d7-9faed1a96936 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:18.762154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 587/1183] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:31:18.767963 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0989df7-8274-46fb-b787-c9e44d24e022 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:18.862530 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0989df7-8274-46fb-b787-c9e44d24e022 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:18.894512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b60647-72af-47f7-82af-1f1facfc00b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:18.895383 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b60647-72af-47f7-82af-1f1facfc00b0 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:18.895660 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b60647-72af-47f7-82af-1f1facfc00b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:18.895930 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b60647-72af-47f7-82af-1f1facfc00b0 None admin] PATH_INFO: `/v3/projects/f9c1a307044b483cb389921363971408` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:18.897081 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b60647-72af-47f7-82af-1f1facfc00b0 None admin] RBAC: Authorizing `identity:get_project(project_id=f9c1a307044b483cb389921363971408)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:18.898762 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b60647-72af-47f7-82af-1f1facfc00b0 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:18.900451 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 597/1184] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:31:18 2022] GET /identity/v3/projects/f9c1a307044b483cb389921363971408 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:18.906321 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b975c5-ed7b-431f-83d7-9faed1a96936 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:18.981911 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:19.003530 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b975c5-ed7b-431f-83d7-9faed1a96936 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:19.041531 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6de7a-4715-49b3-aa77-362dbece5426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:19.042179 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6de7a-4715-49b3-aa77-362dbece5426 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:19.042465 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6de7a-4715-49b3-aa77-362dbece5426 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:19.042744 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6de7a-4715-49b3-aa77-362dbece5426 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:19.043603 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c6de7a-4715-49b3-aa77-362dbece5426 None admin] RBAC: Adding query filter params (role.id=ed1cfa2fcf724a4d84f5acea100930c7, scope.project.id=f9c1a307044b483cb389921363971408, user.id=95c673ae8c444ced91db11beeb64fb94) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:19.044498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c6de7a-4715-49b3-aa77-362dbece5426 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:19.045929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c6de7a-4715-49b3-aa77-362dbece5426 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:19.051298 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 588/1185] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:31:18 2022] GET /identity/v3/role_assignments?user.id=95c673ae8c444ced91db11beeb64fb94&scope.project.id=f9c1a307044b483cb389921363971408&role.id=ed1cfa2fcf724a4d84f5acea100930c7 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:21.134113 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71424176-15bd-48aa-b6ca-88474eed896b None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:21.134652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71424176-15bd-48aa-b6ca-88474eed896b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:21.134652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71424176-15bd-48aa-b6ca-88474eed896b None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:21.136068 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 598/1186] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:31:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:21.141713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bae55-b5ee-4f27-b90d-aed400ae15ab None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:21.142105 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bae55-b5ee-4f27-b90d-aed400ae15ab None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:21.142268 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bae55-b5ee-4f27-b90d-aed400ae15ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:21.162182 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-799bae55-b5ee-4f27-b90d-aed400ae15ab None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:21.163403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-799bae55-b5ee-4f27-b90d-aed400ae15ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:21.235794 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 589/1187] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:31:21.258330 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71424176-15bd-48aa-b6ca-88474eed896b None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:21.288481 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:21.356941 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71424176-15bd-48aa-b6ca-88474eed896b None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:21.389927 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b69d1d0-dfef-4e4f-9a04-4be9e2902e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:21.390642 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69d1d0-dfef-4e4f-9a04-4be9e2902e88 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:21.390926 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69d1d0-dfef-4e4f-9a04-4be9e2902e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:21.391230 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69d1d0-dfef-4e4f-9a04-4be9e2902e88 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:21.401224 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b69d1d0-dfef-4e4f-9a04-4be9e2902e88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:21.408791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-0b69d1d0-dfef-4e4f-9a04-4be9e2902e88 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:21.408791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:21.408791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:21.408791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:21.408791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:21.408791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:21.408791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:21.408791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:21.408791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:21.408791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:21.408791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:21.408791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:21.408791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:21.408791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:21.408791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:21.408791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:21.409879 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:21.409879 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:21.409879 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:21.409879 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:21.409879 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:21.409879 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:21.409879 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:21.409879 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:21.409879 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:21.409879 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:21.409879 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:21.409879 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:21.409879 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:21.409879 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:21.409879 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:21.409879 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:21.410752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:21.410752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:21.410752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:21.410752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:21.410752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:21.410752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:21.410752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:21.410752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:21.410752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:21.410752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:21.410752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:21.410752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:31:21.410752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 599/1188] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:31:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:21.415955 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bae55-b5ee-4f27-b90d-aed400ae15ab None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:21.483680 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:21.511655 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bae55-b5ee-4f27-b90d-aed400ae15ab None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:21.544645 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b7b12-88a9-492e-9a60-be6a25429c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:21.545323 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b7b12-88a9-492e-9a60-be6a25429c98 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:21.545602 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b7b12-88a9-492e-9a60-be6a25429c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:21.545873 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b7b12-88a9-492e-9a60-be6a25429c98 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:21.546603 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b7b12-88a9-492e-9a60-be6a25429c98 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:21.546904 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b7b12-88a9-492e-9a60-be6a25429c98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:21.548288 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b7b12-88a9-492e-9a60-be6a25429c98 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:21.555993 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 590/1189] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:21.562255 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b69d1d0-dfef-4e4f-9a04-4be9e2902e88 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:21.648801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:21.658388 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b69d1d0-dfef-4e4f-9a04-4be9e2902e88 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:21.690743 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490cb6b-39d4-4987-9594-aab7f37cb391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:21.691611 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490cb6b-39d4-4987-9594-aab7f37cb391 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:21.691891 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490cb6b-39d4-4987-9594-aab7f37cb391 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:21.692163 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490cb6b-39d4-4987-9594-aab7f37cb391 None admin] PATH_INFO: `/v3/users/95c673ae8c444ced91db11beeb64fb94` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:21.693317 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9490cb6b-39d4-4987-9594-aab7f37cb391 None admin] RBAC: Authorizing `identity:get_user(user_id=95c673ae8c444ced91db11beeb64fb94)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:21.694735 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9490cb6b-39d4-4987-9594-aab7f37cb391 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:21.696358 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 600/1190] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:31:21 2022] GET /identity/v3/users/95c673ae8c444ced91db11beeb64fb94 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:21.701972 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b7b12-88a9-492e-9a60-be6a25429c98 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:21.798275 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b7b12-88a9-492e-9a60-be6a25429c98 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:21.830130 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ef721-a01f-49f9-8a82-0ff1c4bfba55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:21.830977 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ef721-a01f-49f9-8a82-0ff1c4bfba55 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:21.831283 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ef721-a01f-49f9-8a82-0ff1c4bfba55 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:21.831551 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ef721-a01f-49f9-8a82-0ff1c4bfba55 None admin] PATH_INFO: `/v3/projects/f9c1a307044b483cb389921363971408` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:21.832702 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845ef721-a01f-49f9-8a82-0ff1c4bfba55 None admin] RBAC: Authorizing `identity:get_project(project_id=f9c1a307044b483cb389921363971408)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:21.834120 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845ef721-a01f-49f9-8a82-0ff1c4bfba55 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:21.835812 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 591/1191] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:31:21 2022] GET /identity/v3/projects/f9c1a307044b483cb389921363971408 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:21.842025 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490cb6b-39d4-4987-9594-aab7f37cb391 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:21.938661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490cb6b-39d4-4987-9594-aab7f37cb391 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:21.971238 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874cefe-52f3-4cd7-8736-8c0d374a34a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:21.972110 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874cefe-52f3-4cd7-8736-8c0d374a34a0 None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:21.972394 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874cefe-52f3-4cd7-8736-8c0d374a34a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:21.972670 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874cefe-52f3-4cd7-8736-8c0d374a34a0 None admin] PATH_INFO: `/v3/projects/f9c1a307044b483cb389921363971408/users/95c673ae8c444ced91db11beeb64fb94/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:21.974593 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7874cefe-52f3-4cd7-8736-8c0d374a34a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=f9c1a307044b483cb389921363971408, user_id=95c673ae8c444ced91db11beeb64fb94, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:21.976056 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7874cefe-52f3-4cd7-8736-8c0d374a34a0 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:21.988190 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 601/1192] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Jun 15 14:31:21 2022] PUT /identity/v3/projects/f9c1a307044b483cb389921363971408/users/95c673ae8c444ced91db11beeb64fb94/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:31:24.059125 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be4e3d-1ed3-480b-9cbe-6136b44283c2 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:24.059632 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be4e3d-1ed3-480b-9cbe-6136b44283c2 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:24.059724 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be4e3d-1ed3-480b-9cbe-6136b44283c2 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:24.061074 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 592/1193] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:24.066103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b59b1-c0e6-4f8b-8992-96a46938b5cb None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:24.066490 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b59b1-c0e6-4f8b-8992-96a46938b5cb None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:24.066656 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b59b1-c0e6-4f8b-8992-96a46938b5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:24.084336 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-804b59b1-c0e6-4f8b-8992-96a46938b5cb None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:24.085515 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-804b59b1-c0e6-4f8b-8992-96a46938b5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:24.098747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:24.151142 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 602/1194] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:31:24.174513 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be4e3d-1ed3-480b-9cbe-6136b44283c2 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:24.177390 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:24.275302 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be4e3d-1ed3-480b-9cbe-6136b44283c2 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:24.307923 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b7b34-36e9-466f-ae71-8f034d0a5952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:24.308867 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b7b34-36e9-466f-ae71-8f034d0a5952 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:24.309231 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b7b34-36e9-466f-ae71-8f034d0a5952 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:24.309970 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b7b34-36e9-466f-ae71-8f034d0a5952 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:24.319926 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b7b34-36e9-466f-ae71-8f034d0a5952 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:24.328090 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-003b7b34-36e9-466f-ae71-8f034d0a5952 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:24.328090 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:24.328090 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:24.328090 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:24.328090 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:24.328090 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:24.328090 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:24.328090 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:24.328090 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:24.328090 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:24.328090 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:24.328090 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:24.328090 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:24.328090 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:24.328090 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:24.328090 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:24.329184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:24.329184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:24.329184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:24.329184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:24.329184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:24.329184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:24.329184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:24.329184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:24.329184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:24.329184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:24.329184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:24.329184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:24.329184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:24.329184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:24.329184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:24.329184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:24.330203 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:24.330203 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:24.330203 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:24.330203 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:24.330203 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:24.330203 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:24.330203 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:24.330203 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:24.330203 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:24.330203 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:24.330203 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:24.330203 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:31:24.330203 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 593/1195] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:31:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:24.335238 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b59b1-c0e6-4f8b-8992-96a46938b5cb None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:24.429600 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b59b1-c0e6-4f8b-8992-96a46938b5cb None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:24.462143 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcdcc70-d582-4544-bb2e-762a93751333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:24.462797 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcdcc70-d582-4544-bb2e-762a93751333 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:24.463143 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcdcc70-d582-4544-bb2e-762a93751333 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:24.463384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcdcc70-d582-4544-bb2e-762a93751333 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:24.464142 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcdcc70-d582-4544-bb2e-762a93751333 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:24.464452 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcdcc70-d582-4544-bb2e-762a93751333 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:24.465844 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcdcc70-d582-4544-bb2e-762a93751333 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:24.473674 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 603/1196] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:24.479399 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b7b34-36e9-466f-ae71-8f034d0a5952 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:24.539733 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:24.578471 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b7b34-36e9-466f-ae71-8f034d0a5952 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:24.610215 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312e7cf-8599-4892-9d46-6affa5355435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:24.611094 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312e7cf-8599-4892-9d46-6affa5355435 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:24.611375 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312e7cf-8599-4892-9d46-6affa5355435 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:24.611643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312e7cf-8599-4892-9d46-6affa5355435 None admin] PATH_INFO: `/v3/users/95c673ae8c444ced91db11beeb64fb94` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:24.612798 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e312e7cf-8599-4892-9d46-6affa5355435 None admin] RBAC: Authorizing `identity:get_user(user_id=95c673ae8c444ced91db11beeb64fb94)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:24.614211 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e312e7cf-8599-4892-9d46-6affa5355435 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:24.615846 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 594/1197] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Jun 15 14:31:24 2022] GET /identity/v3/users/95c673ae8c444ced91db11beeb64fb94 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:24.621508 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcdcc70-d582-4544-bb2e-762a93751333 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:24.634279 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:24.719070 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcdcc70-d582-4544-bb2e-762a93751333 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:24.752738 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4a55c-c055-49ff-9f70-a18c79b0ee20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:24.753387 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4a55c-c055-49ff-9f70-a18c79b0ee20 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:24.753671 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4a55c-c055-49ff-9f70-a18c79b0ee20 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:24.753946 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4a55c-c055-49ff-9f70-a18c79b0ee20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:24.819300 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c4a55c-c055-49ff-9f70-a18c79b0ee20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:24.820819 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c4a55c-c055-49ff-9f70-a18c79b0ee20 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:24.831478 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:24.915605 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 604/1198] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:31:24.921892 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312e7cf-8599-4892-9d46-6affa5355435 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:25.017631 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312e7cf-8599-4892-9d46-6affa5355435 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:25.045371 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:25.051391 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9dbff-40b7-49a7-9762-3651b9f5fdfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:25.052249 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9dbff-40b7-49a7-9762-3651b9f5fdfc None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:25.052536 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9dbff-40b7-49a7-9762-3651b9f5fdfc None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:25.052815 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9dbff-40b7-49a7-9762-3651b9f5fdfc None admin] PATH_INFO: `/v3/projects/f9c1a307044b483cb389921363971408` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:25.053988 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c9dbff-40b7-49a7-9762-3651b9f5fdfc None admin] RBAC: Authorizing `identity:get_project(project_id=f9c1a307044b483cb389921363971408)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:25.055459 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c9dbff-40b7-49a7-9762-3651b9f5fdfc None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:25.057182 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 595/1199] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Jun 15 14:31:24 2022] GET /identity/v3/projects/f9c1a307044b483cb389921363971408 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:25.063025 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4a55c-c055-49ff-9f70-a18c79b0ee20 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:25.158583 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4a55c-c055-49ff-9f70-a18c79b0ee20 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:25.169433 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:25.192626 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21894f63-2d26-419f-99c1-973b24f709d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:25.193266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21894f63-2d26-419f-99c1-973b24f709d7 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:25.193549 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21894f63-2d26-419f-99c1-973b24f709d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:25.193827 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21894f63-2d26-419f-99c1-973b24f709d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:25.194644 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21894f63-2d26-419f-99c1-973b24f709d7 None admin] RBAC: Adding query filter params (role.id=ed1cfa2fcf724a4d84f5acea100930c7, scope.project.id=f9c1a307044b483cb389921363971408, user.id=95c673ae8c444ced91db11beeb64fb94) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:25.194962 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21894f63-2d26-419f-99c1-973b24f709d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:25.196401 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21894f63-2d26-419f-99c1-973b24f709d7 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:25.203809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 605/1200] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:31:25 2022] GET /identity/v3/role_assignments?user.id=95c673ae8c444ced91db11beeb64fb94&scope.project.id=f9c1a307044b483cb389921363971408&role.id=ed1cfa2fcf724a4d84f5acea100930c7 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:35.049987 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:35.199505 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=13 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:21.207414 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900443a-e583-4c25-9e2d-38844963d2f3 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:21.208015 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900443a-e583-4c25-9e2d-38844963d2f3 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:21.208015 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900443a-e583-4c25-9e2d-38844963d2f3 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:21.209347 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 596/1201] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:21.215423 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f1396-76f3-4d4d-83d1-cc632c868f51 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:21.215868 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f1396-76f3-4d4d-83d1-cc632c868f51 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:21.215974 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f1396-76f3-4d4d-83d1-cc632c868f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:21.235110 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-902f1396-76f3-4d4d-83d1-cc632c868f51 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:21.236302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-902f1396-76f3-4d4d-83d1-cc632c868f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:21.315864 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 606/1202] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:21.341812 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900443a-e583-4c25-9e2d-38844963d2f3 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:21.470764 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900443a-e583-4c25-9e2d-38844963d2f3 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:21.511702 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f011f9-3844-4bc2-b847-625d3c1859c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:21.512529 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f011f9-3844-4bc2-b847-625d3c1859c3 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:21.512815 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f011f9-3844-4bc2-b847-625d3c1859c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:21.513665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f011f9-3844-4bc2-b847-625d3c1859c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:21.524643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f011f9-3844-4bc2-b847-625d3c1859c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:21.526372 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f011f9-3844-4bc2-b847-625d3c1859c3 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:21.553027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-09f011f9-3844-4bc2-b847-625d3c1859c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:21.553027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:21.553027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:21.553027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:21.553027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:21.553027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:21.553027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:21.553027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:21.553027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:21.553027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:21.553027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:21.553027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:21.553027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:21.553027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:21.553027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:21.554070 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:21.554070 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:21.554070 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:21.554070 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:21.554070 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:21.554070 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:21.554070 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:21.554070 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:21.554070 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:21.554070 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:21.554070 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:21.554070 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:21.554070 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:21.554070 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:21.554070 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:21.554070 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:21.554070 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:21.555080 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:21.555080 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:21.555080 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:21.555080 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:32:21.555080 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 597/1203] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:21.561112 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f1396-76f3-4d4d-83d1-cc632c868f51 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:21.689801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f1396-76f3-4d4d-83d1-cc632c868f51 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:21.721820 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:21.732284 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa81e6a-16ec-4bb2-9ed3-b393ca73b121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:21.733041 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa81e6a-16ec-4bb2-9ed3-b393ca73b121 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:21.733546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa81e6a-16ec-4bb2-9ed3-b393ca73b121 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:21.734024 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa81e6a-16ec-4bb2-9ed3-b393ca73b121 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:21.734795 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa81e6a-16ec-4bb2-9ed3-b393ca73b121 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:21.735096 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa81e6a-16ec-4bb2-9ed3-b393ca73b121 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:21.736481 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa81e6a-16ec-4bb2-9ed3-b393ca73b121 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:21.753453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 607/1204] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:21.754924 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f011f9-3844-4bc2-b847-625d3c1859c3 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:21.830574 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:21.899488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f011f9-3844-4bc2-b847-625d3c1859c3 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:21.952366 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf6b4a1-a12a-4b1a-9156-f53d7b8fcdb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:21.953087 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6b4a1-a12a-4b1a-9156-f53d7b8fcdb1 None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:21.953989 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6b4a1-a12a-4b1a-9156-f53d7b8fcdb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:21.954277 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6b4a1-a12a-4b1a-9156-f53d7b8fcdb1 None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:21.955159 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf6b4a1-a12a-4b1a-9156-f53d7b8fcdb1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:21.956637 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf6b4a1-a12a-4b1a-9156-f53d7b8fcdb1 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:21.981153 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 598/1205] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Jun 15 14:32:21 2022] POST /identity/v3/users => generated 251 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:32:24.228076 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71d245-b3f1-4a34-a48e-744dd52fe577 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:24.228645 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71d245-b3f1-4a34-a48e-744dd52fe577 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:24.228645 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71d245-b3f1-4a34-a48e-744dd52fe577 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:24.230185 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 608/1206] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:32:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:24.235675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15027296-735b-4867-8dd1-870576f6a9ad None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:24.236103 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15027296-735b-4867-8dd1-870576f6a9ad None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:24.236293 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15027296-735b-4867-8dd1-870576f6a9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:24.253773 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-15027296-735b-4867-8dd1-870576f6a9ad None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:24.254937 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-15027296-735b-4867-8dd1-870576f6a9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:24.332298 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 599/1207] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:24.357879 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71d245-b3f1-4a34-a48e-744dd52fe577 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:24.474394 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71d245-b3f1-4a34-a48e-744dd52fe577 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:24.526566 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e5580-6373-4bbb-92fe-3bfc92491db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:24.527364 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e5580-6373-4bbb-92fe-3bfc92491db1 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:24.527702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e5580-6373-4bbb-92fe-3bfc92491db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:24.527971 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e5580-6373-4bbb-92fe-3bfc92491db1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:24.540575 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23e5580-6373-4bbb-92fe-3bfc92491db1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:32:24.548313 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-e23e5580-6373-4bbb-92fe-3bfc92491db1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:32:24.548313 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:24.548313 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:24.548313 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:24.548313 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:24.548313 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:24.548313 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:24.548313 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:24.548313 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:24.548313 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:24.548313 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:24.548313 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:24.548313 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:32:24.548313 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:32:24.548313 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:32:24.548313 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:32:24.550458 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:32:24.550458 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:32:24.550458 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:32:24.550458 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:32:24.550458 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:24.550458 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:24.550458 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:24.550458 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:24.550458 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:24.550458 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:24.550458 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:24.550458 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:24.550458 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:24.550458 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:24.550458 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:24.550458 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:24.551345 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:24.551345 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:24.551345 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:24.551345 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:24.551345 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:32:24.551345 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:32:24.551345 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:32:24.551345 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:32:24.551345 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:32:24.551345 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:32:24.551345 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:32:24.551345 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:32:24.551345 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 609/1208] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Jun 15 14:32:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:24.555680 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15027296-735b-4867-8dd1-870576f6a9ad None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:24.603612 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:24.673497 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15027296-735b-4867-8dd1-870576f6a9ad None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:24.710387 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0eae6b-5ccd-4843-8fa4-07d8ae629977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:24.711160 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0eae6b-5ccd-4843-8fa4-07d8ae629977 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:24.711529 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0eae6b-5ccd-4843-8fa4-07d8ae629977 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:24.711759 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0eae6b-5ccd-4843-8fa4-07d8ae629977 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:24.712506 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0eae6b-5ccd-4843-8fa4-07d8ae629977 None admin] RBAC: Adding query filter params (name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:24.712863 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0eae6b-5ccd-4843-8fa4-07d8ae629977 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:24.714257 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0eae6b-5ccd-4843-8fa4-07d8ae629977 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:24.725568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 600/1209] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Jun 15 14:32:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:24.732338 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e5580-6373-4bbb-92fe-3bfc92491db1 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:24.796095 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:24.859335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e5580-6373-4bbb-92fe-3bfc92491db1 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:24.920115 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ae5c9-9047-4627-958b-38e572044895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:24.920928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ae5c9-9047-4627-958b-38e572044895 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:24.921277 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ae5c9-9047-4627-958b-38e572044895 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:24.921586 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ae5c9-9047-4627-958b-38e572044895 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:24.935836 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449ae5c9-9047-4627-958b-38e572044895 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:24.937735 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449ae5c9-9047-4627-958b-38e572044895 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:24.963246 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-449ae5c9-9047-4627-958b-38e572044895 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:24.963246 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:24.963246 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:24.963246 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:24.963246 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:24.963246 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:24.963246 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:24.963246 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:24.963246 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:24.963246 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:24.963246 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:24.963246 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:24.963246 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:24.963246 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:24.963246 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:24.964394 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:24.964394 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:24.964394 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:24.964394 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:24.964394 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:24.964394 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:24.964394 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:24.964394 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:24.964394 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:24.964394 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:24.964394 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:24.964394 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:24.964394 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:24.964394 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:24.964394 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:24.964394 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:24.964394 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:24.966390 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:24.966390 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:24.966390 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:24.966390 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:32:24.966390 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 610/1210] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:24.972940 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0eae6b-5ccd-4843-8fa4-07d8ae629977 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:25.113675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:25.123387 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0eae6b-5ccd-4843-8fa4-07d8ae629977 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:25.189990 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ffe822-59cf-4b54-851d-716c3109baa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:25.191080 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ffe822-59cf-4b54-851d-716c3109baa8 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:25.191552 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ffe822-59cf-4b54-851d-716c3109baa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:25.192050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ffe822-59cf-4b54-851d-716c3109baa8 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:25.193419 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ffe822-59cf-4b54-851d-716c3109baa8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:25.194008 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ffe822-59cf-4b54-851d-716c3109baa8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:25.196940 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ffe822-59cf-4b54-851d-716c3109baa8 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:25.213730 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 601/1211] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:25.220027 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ae5c9-9047-4627-958b-38e572044895 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:25.350697 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ae5c9-9047-4627-958b-38e572044895 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:25.352925 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:25.397202 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a5702-4cd1-44e4-9ec4-bee27b2b3f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:25.397202 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a5702-4cd1-44e4-9ec4-bee27b2b3f20 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:25.397202 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a5702-4cd1-44e4-9ec4-bee27b2b3f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:25.397202 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a5702-4cd1-44e4-9ec4-bee27b2b3f20 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:25.410220 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a5702-4cd1-44e4-9ec4-bee27b2b3f20 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:25.412153 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a5702-4cd1-44e4-9ec4-bee27b2b3f20 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:25.429934 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-c44a5702-4cd1-44e4-9ec4-bee27b2b3f20 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:32:25.429934 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:25.429934 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:25.429934 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:25.429934 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:25.429934 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:25.429934 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:25.429934 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:25.429934 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:25.429934 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:25.429934 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:25.429934 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:25.429934 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:32:25.429934 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:32:25.429934 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:32:25.429934 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:32:25.430954 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:25.430954 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:25.430954 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:32:25.430954 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:25.430954 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:32:25.430954 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:25.430954 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:25.430954 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:25.430954 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:25.430954 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:25.430954 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:25.430954 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:25.430954 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:25.430954 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:25.430954 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:25.430954 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:25.430954 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:25.430954 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:25.437201 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:25.437201 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:25.437201 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:32:25.437201 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:32:25.437201 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:32:25.437201 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:32:25.437201 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:32:25.437201 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:32:25.437201 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:32:25.437201 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:32:25.437201 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 611/1212] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:25.437950 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ffe822-59cf-4b54-851d-716c3109baa8 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:25.581149 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ffe822-59cf-4b54-851d-716c3109baa8 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:25.632644 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea28c71-0c25-411c-be90-709d54b6228a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:25.633519 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea28c71-0c25-411c-be90-709d54b6228a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:25.633993 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea28c71-0c25-411c-be90-709d54b6228a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:25.634394 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea28c71-0c25-411c-be90-709d54b6228a None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:25.635424 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea28c71-0c25-411c-be90-709d54b6228a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:25.635874 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea28c71-0c25-411c-be90-709d54b6228a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:25.637708 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea28c71-0c25-411c-be90-709d54b6228a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:25.663126 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 602/1213] 10.222.0.45 () {56 vars in 1143 bytes} [Wed Jun 15 14:32:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:25.669574 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a5702-4cd1-44e4-9ec4-bee27b2b3f20 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:25.824012 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a5702-4cd1-44e4-9ec4-bee27b2b3f20 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:25.864669 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d084d9e-b79a-4bb0-b34e-aa4bb47351d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:25.865402 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d084d9e-b79a-4bb0-b34e-aa4bb47351d8 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:25.865710 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d084d9e-b79a-4bb0-b34e-aa4bb47351d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:25.865976 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d084d9e-b79a-4bb0-b34e-aa4bb47351d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:25.913811 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:25.967873 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d084d9e-b79a-4bb0-b34e-aa4bb47351d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:25.969521 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d084d9e-b79a-4bb0-b34e-aa4bb47351d8 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:26.118274 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 612/1214] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:32:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:32:26.128454 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea28c71-0c25-411c-be90-709d54b6228a None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:26.165030 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:26.280108 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea28c71-0c25-411c-be90-709d54b6228a None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:26.329064 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b5f57-234a-4685-a205-8f2ef5436fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:26.330431 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b5f57-234a-4685-a205-8f2ef5436fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:26.330858 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b5f57-234a-4685-a205-8f2ef5436fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:26.331344 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b5f57-234a-4685-a205-8f2ef5436fd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:26.347000 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b5f57-234a-4685-a205-8f2ef5436fd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:26.349001 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b5f57-234a-4685-a205-8f2ef5436fd2 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:26.375530 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-a80b5f57-234a-4685-a205-8f2ef5436fd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:26.375530 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:26.375530 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:26.375530 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:26.375530 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:26.375530 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:26.375530 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:26.375530 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:26.375530 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:26.375530 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:26.375530 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:26.375530 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:26.375530 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:26.375530 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:26.375530 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:26.377608 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:26.377608 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:26.377608 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:26.377608 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:26.377608 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:26.377608 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:26.377608 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:26.377608 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:26.377608 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:26.377608 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:26.377608 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:26.377608 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:26.377608 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:26.377608 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:26.377608 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:26.377608 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:26.377608 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:26.378716 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:26.378716 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:26.378716 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:26.378716 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:32:26.378716 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 603/1215] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:26.383645 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d084d9e-b79a-4bb0-b34e-aa4bb47351d8 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:26.474629 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:26.529421 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d084d9e-b79a-4bb0-b34e-aa4bb47351d8 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:26.570937 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e68539e-a727-4e0f-8be1-f7449f07463d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:26.571643 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68539e-a727-4e0f-8be1-f7449f07463d None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:26.571925 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68539e-a727-4e0f-8be1-f7449f07463d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:26.572254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68539e-a727-4e0f-8be1-f7449f07463d None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:26.572928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e68539e-a727-4e0f-8be1-f7449f07463d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:26.573270 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e68539e-a727-4e0f-8be1-f7449f07463d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:26.574647 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e68539e-a727-4e0f-8be1-f7449f07463d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:26.587875 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 613/1216] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:26.594004 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b5f57-234a-4685-a205-8f2ef5436fd2 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:26.686110 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:26.733762 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b5f57-234a-4685-a205-8f2ef5436fd2 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:26.779957 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0310b8-7ec6-4e37-a69d-bdd2bc7725b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:26.780924 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0310b8-7ec6-4e37-a69d-bdd2bc7725b1 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:26.781220 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0310b8-7ec6-4e37-a69d-bdd2bc7725b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:26.781490 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0310b8-7ec6-4e37-a69d-bdd2bc7725b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:26.789801 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0310b8-7ec6-4e37-a69d-bdd2bc7725b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:26.791381 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0310b8-7ec6-4e37-a69d-bdd2bc7725b1 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:26.807579 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-ee0310b8-7ec6-4e37-a69d-bdd2bc7725b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:26.807579 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:26.807579 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:26.807579 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:26.807579 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:26.807579 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:26.807579 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:26.807579 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:26.807579 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:26.807579 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:26.807579 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:26.807579 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:26.807579 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:32:26.807579 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:32:26.807579 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:32:26.808592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:32:26.808592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:26.808592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:26.808592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:26.808592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:26.808592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:26.808592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:26.808592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:26.808592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:26.808592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:26.808592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:26.808592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:26.808592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:26.808592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:26.808592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:26.808592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:26.808592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:26.809480 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:32:26.809480 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:32:26.809480 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:32:26.809480 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:32:26.809480 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:32:26.809480 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:32:26.809480 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:26.809480 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:32:26.809480 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 604/1217] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:32:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:26.815013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e68539e-a727-4e0f-8be1-f7449f07463d None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:26.927895 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e68539e-a727-4e0f-8be1-f7449f07463d None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:26.951798 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:26.965111 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc033f6-4ddd-43dd-8e58-9b2f950c2177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:26.967013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc033f6-4ddd-43dd-8e58-9b2f950c2177 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:26.967013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc033f6-4ddd-43dd-8e58-9b2f950c2177 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:26.967548 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc033f6-4ddd-43dd-8e58-9b2f950c2177 None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:26.968615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc033f6-4ddd-43dd-8e58-9b2f950c2177 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:26.969126 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc033f6-4ddd-43dd-8e58-9b2f950c2177 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:26.970704 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc033f6-4ddd-43dd-8e58-9b2f950c2177 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:26.988497 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 614/1218] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:32:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:26.990579 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0310b8-7ec6-4e37-a69d-bdd2bc7725b1 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:27.125871 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0310b8-7ec6-4e37-a69d-bdd2bc7725b1 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:27.161661 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:27.168622 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae72afa-23f6-45cf-8e16-d5e9fffc9f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:27.169924 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae72afa-23f6-45cf-8e16-d5e9fffc9f58 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:27.170308 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae72afa-23f6-45cf-8e16-d5e9fffc9f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:27.170706 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae72afa-23f6-45cf-8e16-d5e9fffc9f58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:27.171679 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae72afa-23f6-45cf-8e16-d5e9fffc9f58 None admin] RBAC: Adding query filter params (role.id=13dcf68cca9c44888c0fd27a78db3dbe, scope.project.id=d488c79cb1704e9496fc196d8aa6c104, user.id=f4128111a9aa450abb2146a5d9926dec) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:27.172086 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae72afa-23f6-45cf-8e16-d5e9fffc9f58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:27.173710 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae72afa-23f6-45cf-8e16-d5e9fffc9f58 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:27.191032 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 605/1219] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:32:26 2022] GET /identity/v3/role_assignments?user.id=f4128111a9aa450abb2146a5d9926dec&scope.project.id=d488c79cb1704e9496fc196d8aa6c104&role.id=13dcf68cca9c44888c0fd27a78db3dbe => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:29.488030 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733afd3-0976-4ce7-a2b9-b729f3bb247e None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:29.488731 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733afd3-0976-4ce7-a2b9-b729f3bb247e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:29.488830 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733afd3-0976-4ce7-a2b9-b729f3bb247e None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:29.490422 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 615/1220] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:32:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:32:29.497931 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26ddad-52bc-417a-b15b-ce1f84d96b76 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:29.498466 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26ddad-52bc-417a-b15b-ce1f84d96b76 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:29.498605 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26ddad-52bc-417a-b15b-ce1f84d96b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:29.522225 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-ec26ddad-52bc-417a-b15b-ce1f84d96b76 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:29.524296 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ec26ddad-52bc-417a-b15b-ce1f84d96b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:29.604750 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 606/1221] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:32:29.632730 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733afd3-0976-4ce7-a2b9-b729f3bb247e None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:29.761959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733afd3-0976-4ce7-a2b9-b729f3bb247e None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:29.806451 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ca2ec-feab-4068-adba-c136573aca49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:29.807287 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ca2ec-feab-4068-adba-c136573aca49 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:29.807568 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ca2ec-feab-4068-adba-c136573aca49 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:29.807850 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ca2ec-feab-4068-adba-c136573aca49 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:29.818804 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5ca2ec-feab-4068-adba-c136573aca49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:32:29.828636 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-2d5ca2ec-feab-4068-adba-c136573aca49 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:32:29.828636 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:29.828636 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:29.828636 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:29.828636 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:29.828636 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:29.828636 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:29.828636 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:29.828636 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:29.828636 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:29.828636 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:29.828636 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:29.828636 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:32:29.828636 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:32:29.828636 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:32:29.828636 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:32:29.829685 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:32:29.829685 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:32:29.829685 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:32:29.829685 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:32:29.829685 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:29.829685 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:29.829685 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:29.829685 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:29.829685 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:29.829685 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:29.829685 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:29.829685 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:29.829685 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:29.829685 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:29.829685 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:29.829685 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:29.830514 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:29.830514 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:29.830514 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:29.830514 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:29.830514 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:32:29.830514 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:32:29.830514 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:32:29.830514 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:32:29.830514 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:32:29.830514 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:32:29.830514 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:32:29.830514 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:32:29.830514 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 616/1222] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Jun 15 14:32:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:29.837420 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26ddad-52bc-417a-b15b-ce1f84d96b76 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:29.959734 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26ddad-52bc-417a-b15b-ce1f84d96b76 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:29.995487 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:30.008748 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84005751-2fa0-4101-a24b-d71e776b18ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:30.009422 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84005751-2fa0-4101-a24b-d71e776b18ad None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:30.009705 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84005751-2fa0-4101-a24b-d71e776b18ad None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:30.009983 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84005751-2fa0-4101-a24b-d71e776b18ad None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:30.010728 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84005751-2fa0-4101-a24b-d71e776b18ad None admin] RBAC: Adding query filter params (name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:30.011072 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84005751-2fa0-4101-a24b-d71e776b18ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:30.012489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84005751-2fa0-4101-a24b-d71e776b18ad None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:30.021322 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 607/1223] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Jun 15 14:32:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:30.027558 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ca2ec-feab-4068-adba-c136573aca49 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:30.086314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:30.152927 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ca2ec-feab-4068-adba-c136573aca49 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:30.189438 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55b333a-a40c-4cb6-8b15-a45b181a7631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:30.190322 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b333a-a40c-4cb6-8b15-a45b181a7631 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:30.190609 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b333a-a40c-4cb6-8b15-a45b181a7631 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:30.190903 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b333a-a40c-4cb6-8b15-a45b181a7631 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:30.203545 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55b333a-a40c-4cb6-8b15-a45b181a7631 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:30.205012 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55b333a-a40c-4cb6-8b15-a45b181a7631 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:30.225196 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-c55b333a-a40c-4cb6-8b15-a45b181a7631 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:30.225196 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:30.225196 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:30.225196 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:30.225196 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:30.225196 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:30.225196 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:30.225196 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:30.225196 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:30.225196 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:30.225196 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:30.225196 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:30.225196 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:30.225196 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:30.225196 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:30.226306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:30.226306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:30.226306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:30.226306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:30.226306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:30.226306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:30.226306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:30.226306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:30.226306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:30.226306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:30.226306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:30.226306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:30.226306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:30.226306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:30.226306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:30.226306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:30.226306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:30.232524 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:30.232524 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:30.232524 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:30.232524 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:32:30.232524 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 617/1224] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:30.233083 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84005751-2fa0-4101-a24b-d71e776b18ad None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:30.369091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84005751-2fa0-4101-a24b-d71e776b18ad None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:30.425975 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a842c-2c7e-467f-85fd-63b8638ba963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:30.426916 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a842c-2c7e-467f-85fd-63b8638ba963 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:30.427240 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a842c-2c7e-467f-85fd-63b8638ba963 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:30.428570 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a842c-2c7e-467f-85fd-63b8638ba963 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:30.429605 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44a842c-2c7e-467f-85fd-63b8638ba963 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:30.430982 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44a842c-2c7e-467f-85fd-63b8638ba963 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:30.432433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44a842c-2c7e-467f-85fd-63b8638ba963 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:30.455966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 608/1225] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:30.464956 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55b333a-a40c-4cb6-8b15-a45b181a7631 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:30.605584 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:30.613193 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55b333a-a40c-4cb6-8b15-a45b181a7631 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:30.653639 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232822b5-d724-4efd-a97a-15bb94561fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:30.654406 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232822b5-d724-4efd-a97a-15bb94561fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:30.654790 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232822b5-d724-4efd-a97a-15bb94561fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:30.656249 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232822b5-d724-4efd-a97a-15bb94561fe0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:30.665427 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232822b5-d724-4efd-a97a-15bb94561fe0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:30.667820 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232822b5-d724-4efd-a97a-15bb94561fe0 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:30.688288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-232822b5-d724-4efd-a97a-15bb94561fe0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:32:30.688288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:30.688288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:30.688288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:30.688288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:30.688288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:30.688288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:30.688288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:30.688288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:30.688288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:30.688288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:30.688288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:30.688288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:32:30.688288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:32:30.688288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:32:30.688288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:32:30.689525 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:30.689525 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:30.689525 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:32:30.689525 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:30.689525 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:32:30.689525 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:30.689525 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:30.689525 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:30.689525 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:30.689525 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:30.689525 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:30.689525 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:30.689525 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:30.689525 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:30.689525 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:30.689525 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:30.689525 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:30.689525 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:30.690796 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:30.690796 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:30.690796 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:32:30.690796 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:32:30.690796 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:32:30.690796 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:32:30.690796 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:32:30.690796 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:32:30.690796 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:32:30.690796 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:32:30.690796 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 618/1226] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:32:30.696882 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a842c-2c7e-467f-85fd-63b8638ba963 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:30.767902 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:30.852613 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a842c-2c7e-467f-85fd-63b8638ba963 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:30.914030 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac29338-6c35-4b18-8641-2675d03c111f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:30.915047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac29338-6c35-4b18-8641-2675d03c111f None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:30.915476 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac29338-6c35-4b18-8641-2675d03c111f None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:30.915770 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac29338-6c35-4b18-8641-2675d03c111f None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:30.916892 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac29338-6c35-4b18-8641-2675d03c111f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:30.917257 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac29338-6c35-4b18-8641-2675d03c111f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:30.920304 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac29338-6c35-4b18-8641-2675d03c111f None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:30.950960 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 609/1227] 10.222.0.45 () {56 vars in 1143 bytes} [Wed Jun 15 14:32:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:30.952409 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232822b5-d724-4efd-a97a-15bb94561fe0 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:31.084661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232822b5-d724-4efd-a97a-15bb94561fe0 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:31.131689 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0cc4c8-3acd-4cfe-9d0b-a5a2bdfcfd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:31.132583 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0cc4c8-3acd-4cfe-9d0b-a5a2bdfcfd3d None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:31.132864 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0cc4c8-3acd-4cfe-9d0b-a5a2bdfcfd3d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:31.133147 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0cc4c8-3acd-4cfe-9d0b-a5a2bdfcfd3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:31.152425 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0cc4c8-3acd-4cfe-9d0b-a5a2bdfcfd3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:31.154977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0cc4c8-3acd-4cfe-9d0b-a5a2bdfcfd3d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:31.183450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-7b0cc4c8-3acd-4cfe-9d0b-a5a2bdfcfd3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:31.183450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:31.183450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:31.183450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:31.183450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:31.183450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:31.183450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:31.183450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:31.183450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:31.183450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:31.183450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:31.183450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:31.183450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:31.183450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:31.183450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:31.184476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:31.184476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:31.184476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:31.184476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:31.184476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:31.184476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:31.184476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:31.184476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:31.184476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:31.184476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:31.184476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:31.184476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:31.184476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:31.184476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:31.184476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:31.184476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:31.184476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:31.185452 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:31.185452 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:31.185452 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:31.185452 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:32:31.185452 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 619/1228] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:31.190750 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac29338-6c35-4b18-8641-2675d03c111f None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:31.337418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac29338-6c35-4b18-8641-2675d03c111f None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:31.340071 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:31.377672 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564bcaa7-e00f-4353-84a4-dc54bc7b98c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:31.378398 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bcaa7-e00f-4353-84a4-dc54bc7b98c7 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:31.378751 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bcaa7-e00f-4353-84a4-dc54bc7b98c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:31.379060 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bcaa7-e00f-4353-84a4-dc54bc7b98c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:31.379798 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564bcaa7-e00f-4353-84a4-dc54bc7b98c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:31.380115 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564bcaa7-e00f-4353-84a4-dc54bc7b98c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:31.381492 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564bcaa7-e00f-4353-84a4-dc54bc7b98c7 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:31.394953 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 610/1229] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:31.401222 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0cc4c8-3acd-4cfe-9d0b-a5a2bdfcfd3d None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:31.411273 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:31.547227 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0cc4c8-3acd-4cfe-9d0b-a5a2bdfcfd3d None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:31.595427 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c947c1-c707-44fa-92e7-54495cd192e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:31.596880 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c947c1-c707-44fa-92e7-54495cd192e1 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:31.597388 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c947c1-c707-44fa-92e7-54495cd192e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:31.597741 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c947c1-c707-44fa-92e7-54495cd192e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:31.610549 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c947c1-c707-44fa-92e7-54495cd192e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:31.613251 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c947c1-c707-44fa-92e7-54495cd192e1 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:31.639462 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-f3c947c1-c707-44fa-92e7-54495cd192e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:31.639462 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:31.639462 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:31.639462 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:31.639462 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:31.639462 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:31.639462 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:31.639462 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:31.639462 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:31.639462 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:31.639462 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:31.639462 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:31.639462 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:32:31.639462 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:32:31.639462 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:32:31.645596 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:32:31.645596 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:31.645596 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:31.645596 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:31.645596 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:31.645596 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:31.645596 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:31.645596 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:31.645596 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:31.645596 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:31.645596 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:31.645596 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:31.645596 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:31.645596 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:31.645596 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:31.645596 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:31.645596 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:31.650858 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:32:31.650858 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:32:31.650858 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:32:31.650858 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:32:31.650858 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:32:31.650858 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:32:31.650858 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:31.650858 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:32:31.650858 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 620/1230] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:32:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:31.652389 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564bcaa7-e00f-4353-84a4-dc54bc7b98c7 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:31.793908 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564bcaa7-e00f-4353-84a4-dc54bc7b98c7 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:31.842740 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc1cb6-74fa-462f-9d97-4d6f6ff88dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:31.844720 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc1cb6-74fa-462f-9d97-4d6f6ff88dee None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:31.845470 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc1cb6-74fa-462f-9d97-4d6f6ff88dee None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:31.846521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc1cb6-74fa-462f-9d97-4d6f6ff88dee None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:31.847545 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fc1cb6-74fa-462f-9d97-4d6f6ff88dee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:31.848074 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fc1cb6-74fa-462f-9d97-4d6f6ff88dee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:31.849935 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fc1cb6-74fa-462f-9d97-4d6f6ff88dee None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:31.868798 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 611/1231] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:32:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:31.876429 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c947c1-c707-44fa-92e7-54495cd192e1 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:31.972580 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:32.019356 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c947c1-c707-44fa-92e7-54495cd192e1 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:32.070627 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c113481-f633-40bf-a845-b212b4c0bc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:32.073218 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c113481-f633-40bf-a845-b212b4c0bc2c None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:32.073814 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c113481-f633-40bf-a845-b212b4c0bc2c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:32.074325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c113481-f633-40bf-a845-b212b4c0bc2c None admin] PATH_INFO: `/v3/projects/d488c79cb1704e9496fc196d8aa6c104/users/f4128111a9aa450abb2146a5d9926dec/roles/13dcf68cca9c44888c0fd27a78db3dbe` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:32.107656 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c113481-f633-40bf-a845-b212b4c0bc2c None admin] RBAC: Authorizing `identity:create_grant(project_id=d488c79cb1704e9496fc196d8aa6c104, user_id=f4128111a9aa450abb2146a5d9926dec, role_id=13dcf68cca9c44888c0fd27a78db3dbe)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:32.109409 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c113481-f633-40bf-a845-b212b4c0bc2c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:32.125767 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 621/1232] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Jun 15 14:32:31 2022] PUT /identity/v3/projects/d488c79cb1704e9496fc196d8aa6c104/users/f4128111a9aa450abb2146a5d9926dec/roles/13dcf68cca9c44888c0fd27a78db3dbe => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:32:34.286432 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b979a0-c6ef-4333-8c98-44e72c04f386 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:34.286980 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b979a0-c6ef-4333-8c98-44e72c04f386 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:34.286980 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b979a0-c6ef-4333-8c98-44e72c04f386 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:34.288495 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 612/1233] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:32:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:34.293747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831964c-e12e-40dd-8c2d-624968458601 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:34.294165 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831964c-e12e-40dd-8c2d-624968458601 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:34.294310 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831964c-e12e-40dd-8c2d-624968458601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:34.312136 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-6831964c-e12e-40dd-8c2d-624968458601 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:34.313504 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6831964c-e12e-40dd-8c2d-624968458601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:34.386984 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 622/1234] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:34.412935 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b979a0-c6ef-4333-8c98-44e72c04f386 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:34.440045 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:34.536074 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b979a0-c6ef-4333-8c98-44e72c04f386 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:34.571737 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e506e-4036-454f-9030-b1a3ed37f71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:34.572539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e506e-4036-454f-9030-b1a3ed37f71f None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:34.572822 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e506e-4036-454f-9030-b1a3ed37f71f None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:34.573096 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e506e-4036-454f-9030-b1a3ed37f71f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:34.584572 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61e506e-4036-454f-9030-b1a3ed37f71f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:32:34.593558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-a61e506e-4036-454f-9030-b1a3ed37f71f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:32:34.593558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:34.593558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:34.593558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:34.593558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:34.593558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:34.593558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:34.593558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:34.593558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:34.593558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:34.593558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:34.593558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:34.593558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:32:34.593558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:32:34.593558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:32:34.593558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:32:34.594742 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:32:34.594742 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:32:34.594742 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:32:34.594742 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:32:34.594742 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:34.594742 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:34.594742 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:34.594742 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:34.594742 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:34.594742 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:34.594742 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:34.594742 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:34.594742 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:34.594742 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:34.594742 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:34.594742 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:34.595933 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:34.595933 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:34.595933 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:34.595933 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:34.595933 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:32:34.595933 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:32:34.595933 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:32:34.595933 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:32:34.595933 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:32:34.595933 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:32:34.595933 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:32:34.595933 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:32:34.595933 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 613/1235] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Jun 15 14:32:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:34.601023 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6831964c-e12e-40dd-8c2d-624968458601 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:34.650003 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:34.729342 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6831964c-e12e-40dd-8c2d-624968458601 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:34.773158 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69cd4fe-dd0b-4370-b350-97026af8e7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:34.774124 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69cd4fe-dd0b-4370-b350-97026af8e7c3 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:34.774412 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69cd4fe-dd0b-4370-b350-97026af8e7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:34.775026 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69cd4fe-dd0b-4370-b350-97026af8e7c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:34.776513 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69cd4fe-dd0b-4370-b350-97026af8e7c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:34.776978 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69cd4fe-dd0b-4370-b350-97026af8e7c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:34.778392 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69cd4fe-dd0b-4370-b350-97026af8e7c3 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:34.788422 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 623/1236] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Jun 15 14:32:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:34.796640 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e506e-4036-454f-9030-b1a3ed37f71f None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:34.908024 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:34.933748 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e506e-4036-454f-9030-b1a3ed37f71f None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:34.975340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93173087-c931-4e7f-85d0-528be9d2928f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:34.976207 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93173087-c931-4e7f-85d0-528be9d2928f None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:34.976499 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93173087-c931-4e7f-85d0-528be9d2928f None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:34.976914 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93173087-c931-4e7f-85d0-528be9d2928f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:34.988624 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93173087-c931-4e7f-85d0-528be9d2928f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:34.990211 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93173087-c931-4e7f-85d0-528be9d2928f None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:35.008693 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-93173087-c931-4e7f-85d0-528be9d2928f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:35.008693 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:35.008693 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:35.008693 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:35.008693 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:35.008693 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:35.008693 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:35.008693 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:35.008693 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:35.008693 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:35.008693 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:35.008693 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:35.008693 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:35.008693 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:35.008693 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:35.009775 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:35.009775 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:35.009775 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:35.009775 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:35.009775 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:35.009775 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:35.009775 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:35.009775 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:35.009775 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:35.009775 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:35.009775 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:35.009775 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:35.009775 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:35.009775 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:35.009775 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:35.009775 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:35.009775 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:35.010929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:35.010929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:35.010929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:35.010929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:32:35.010929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 614/1237] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:35.017137 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69cd4fe-dd0b-4370-b350-97026af8e7c3 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:35.137792 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69cd4fe-dd0b-4370-b350-97026af8e7c3 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:35.148344 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:35.179901 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec2d44d-a5bf-4dbd-bd95-1e7c99f3f3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:35.180600 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2d44d-a5bf-4dbd-bd95-1e7c99f3f3d5 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:35.180945 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2d44d-a5bf-4dbd-bd95-1e7c99f3f3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:35.181275 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2d44d-a5bf-4dbd-bd95-1e7c99f3f3d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:35.182079 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec2d44d-a5bf-4dbd-bd95-1e7c99f3f3d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:35.182401 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec2d44d-a5bf-4dbd-bd95-1e7c99f3f3d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:35.183992 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec2d44d-a5bf-4dbd-bd95-1e7c99f3f3d5 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:35.198269 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 624/1238] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:35.203917 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93173087-c931-4e7f-85d0-528be9d2928f None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:35.342969 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93173087-c931-4e7f-85d0-528be9d2928f None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:35.382242 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2cd7f4-e6a2-4e48-9060-ba8da8b9295a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:35.383098 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2cd7f4-e6a2-4e48-9060-ba8da8b9295a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:35.383393 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2cd7f4-e6a2-4e48-9060-ba8da8b9295a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:35.383676 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2cd7f4-e6a2-4e48-9060-ba8da8b9295a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:35.387155 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:35.393962 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2cd7f4-e6a2-4e48-9060-ba8da8b9295a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:35.395780 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2cd7f4-e6a2-4e48-9060-ba8da8b9295a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:35.412222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-7e2cd7f4-e6a2-4e48-9060-ba8da8b9295a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:32:35.412222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:35.412222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:35.412222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:35.412222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:35.412222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:35.412222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:35.412222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:35.412222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:35.412222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:35.412222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:35.412222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:35.412222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:32:35.412222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:32:35.412222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:32:35.412222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:32:35.413455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:35.413455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:35.413455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:32:35.413455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:35.413455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:32:35.413455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:35.413455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:35.413455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:35.413455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:35.413455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:35.413455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:35.413455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:35.413455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:35.413455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:35.413455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:35.413455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:35.413455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:35.413455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:35.419936 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:35.419936 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:35.419936 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:32:35.419936 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:32:35.419936 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:32:35.419936 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:32:35.419936 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:32:35.419936 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:32:35.419936 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:32:35.419936 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:32:35.419936 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 615/1239] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:35.420688 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec2d44d-a5bf-4dbd-bd95-1e7c99f3f3d5 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:35.551451 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec2d44d-a5bf-4dbd-bd95-1e7c99f3f3d5 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:35.595880 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d4a2ad-15e0-4f24-a72d-af29fa8fe854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:35.596558 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d4a2ad-15e0-4f24-a72d-af29fa8fe854 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:35.596841 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d4a2ad-15e0-4f24-a72d-af29fa8fe854 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:35.597118 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d4a2ad-15e0-4f24-a72d-af29fa8fe854 None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:35.597997 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d4a2ad-15e0-4f24-a72d-af29fa8fe854 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:35.598329 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d4a2ad-15e0-4f24-a72d-af29fa8fe854 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:35.599886 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d4a2ad-15e0-4f24-a72d-af29fa8fe854 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:35.619812 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 625/1240] 10.222.0.45 () {56 vars in 1143 bytes} [Wed Jun 15 14:32:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:35.627356 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2cd7f4-e6a2-4e48-9060-ba8da8b9295a None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:35.764409 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2cd7f4-e6a2-4e48-9060-ba8da8b9295a None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:35.801557 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e86b928-eda4-4cea-8f7f-6bf5ff4d97ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:35.802151 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86b928-eda4-4cea-8f7f-6bf5ff4d97ba None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:35.802433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86b928-eda4-4cea-8f7f-6bf5ff4d97ba None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:35.802707 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86b928-eda4-4cea-8f7f-6bf5ff4d97ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:35.872179 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:35.881521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e86b928-eda4-4cea-8f7f-6bf5ff4d97ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:35.883065 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e86b928-eda4-4cea-8f7f-6bf5ff4d97ba None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:35.996307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 616/1241] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:32:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:32:36.004231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d4a2ad-15e0-4f24-a72d-af29fa8fe854 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:36.035961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:36.121003 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d4a2ad-15e0-4f24-a72d-af29fa8fe854 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:36.161068 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75efae-8c3b-4824-87c0-819de6fe044f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:36.161984 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75efae-8c3b-4824-87c0-819de6fe044f None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:36.162266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75efae-8c3b-4824-87c0-819de6fe044f None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:36.162545 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75efae-8c3b-4824-87c0-819de6fe044f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:36.175537 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a75efae-8c3b-4824-87c0-819de6fe044f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:36.177221 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a75efae-8c3b-4824-87c0-819de6fe044f None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:36.197702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-4a75efae-8c3b-4824-87c0-819de6fe044f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:36.197702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:36.197702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:36.197702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:36.197702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:36.197702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:36.197702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:36.197702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:36.197702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:36.197702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:36.197702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:36.197702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:36.197702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:36.197702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:36.197702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:36.199378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:36.199378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:36.199378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:36.199378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:36.199378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:36.199378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:36.199378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:36.199378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:36.199378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:36.199378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:36.199378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:36.199378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:36.199378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:36.199378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:36.199378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:36.199378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:36.199378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:36.205453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:36.205453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:36.205453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:36.205453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:32:36.205453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 626/1242] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:36.206966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e86b928-eda4-4cea-8f7f-6bf5ff4d97ba None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:36.328276 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:36.337822 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e86b928-eda4-4cea-8f7f-6bf5ff4d97ba None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:36.382810 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343abcb-0126-4eac-becf-db55441ba389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:36.383899 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343abcb-0126-4eac-becf-db55441ba389 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:36.384208 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343abcb-0126-4eac-becf-db55441ba389 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:36.384580 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343abcb-0126-4eac-becf-db55441ba389 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:36.385409 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5343abcb-0126-4eac-becf-db55441ba389 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:36.385832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5343abcb-0126-4eac-becf-db55441ba389 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:36.387360 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5343abcb-0126-4eac-becf-db55441ba389 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:36.402041 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 617/1243] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:36.410147 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75efae-8c3b-4824-87c0-819de6fe044f None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:36.513302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:36.541698 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75efae-8c3b-4824-87c0-819de6fe044f None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:36.580564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3602b-2895-4c53-9adb-d8c2d21aee32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:36.581426 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3602b-2895-4c53-9adb-d8c2d21aee32 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:36.581708 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3602b-2895-4c53-9adb-d8c2d21aee32 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:36.581982 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3602b-2895-4c53-9adb-d8c2d21aee32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:36.590060 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf3602b-2895-4c53-9adb-d8c2d21aee32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:36.592097 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf3602b-2895-4c53-9adb-d8c2d21aee32 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:36.608055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-1cf3602b-2895-4c53-9adb-d8c2d21aee32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:36.608055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:36.608055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:36.608055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:36.608055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:36.608055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:36.608055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:36.608055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:36.608055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:36.608055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:36.608055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:36.608055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:36.608055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:32:36.608055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:32:36.608055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:32:36.609212 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:32:36.609212 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:36.609212 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:36.609212 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:36.609212 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:36.609212 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:36.609212 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:36.609212 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:36.609212 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:36.609212 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:36.609212 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:36.609212 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:36.609212 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:36.609212 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:36.609212 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:36.609212 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:36.609212 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:36.610512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:32:36.610512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:32:36.610512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:32:36.610512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:32:36.610512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:32:36.610512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:32:36.610512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:36.610512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:32:36.610512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 627/1244] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:32:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:32:36.615190 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343abcb-0126-4eac-becf-db55441ba389 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:36.735281 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343abcb-0126-4eac-becf-db55441ba389 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:36.775918 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94309d7-5a82-4eda-8b9f-a07a3bd791ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:36.777033 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94309d7-5a82-4eda-8b9f-a07a3bd791ca None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:36.777318 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94309d7-5a82-4eda-8b9f-a07a3bd791ca None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:36.777595 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94309d7-5a82-4eda-8b9f-a07a3bd791ca None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:36.778610 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94309d7-5a82-4eda-8b9f-a07a3bd791ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:36.779581 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94309d7-5a82-4eda-8b9f-a07a3bd791ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:36.781310 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94309d7-5a82-4eda-8b9f-a07a3bd791ca None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:36.796470 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 618/1245] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:32:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:36.803664 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3602b-2895-4c53-9adb-d8c2d21aee32 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:36.932378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3602b-2895-4c53-9adb-d8c2d21aee32 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:36.973074 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a3ad3-16a3-4ca4-a42a-44c5c0ec38df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:36.973791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a3ad3-16a3-4ca4-a42a-44c5c0ec38df None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:36.974140 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a3ad3-16a3-4ca4-a42a-44c5c0ec38df None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:36.974485 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a3ad3-16a3-4ca4-a42a-44c5c0ec38df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:36.975441 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a3ad3-16a3-4ca4-a42a-44c5c0ec38df None admin] RBAC: Adding query filter params (role.id=13dcf68cca9c44888c0fd27a78db3dbe, scope.project.id=d488c79cb1704e9496fc196d8aa6c104, user.id=f4128111a9aa450abb2146a5d9926dec) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:36.975824 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a3ad3-16a3-4ca4-a42a-44c5c0ec38df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:36.977525 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a3ad3-16a3-4ca4-a42a-44c5c0ec38df None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:36.984940 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 628/1246] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:32:36 2022] GET /identity/v3/role_assignments?user.id=f4128111a9aa450abb2146a5d9926dec&scope.project.id=d488c79cb1704e9496fc196d8aa6c104&role.id=13dcf68cca9c44888c0fd27a78db3dbe => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:39.294099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3829d-7f1f-4e6c-8f03-61e7f940ac82 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:39.295253 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3829d-7f1f-4e6c-8f03-61e7f940ac82 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:39.295253 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3829d-7f1f-4e6c-8f03-61e7f940ac82 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:39.296888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 619/1247] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:32:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:39.302602 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857eb98-b185-4af4-94ee-d41f67a343ca None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:39.303071 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857eb98-b185-4af4-94ee-d41f67a343ca None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:39.303219 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857eb98-b185-4af4-94ee-d41f67a343ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:39.309241 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:39.323314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-8857eb98-b185-4af4-94ee-d41f67a343ca None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:39.324765 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-8857eb98-b185-4af4-94ee-d41f67a343ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:39.412963 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 629/1248] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:39.438109 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3829d-7f1f-4e6c-8f03-61e7f940ac82 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:39.456785 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:39.567581 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3829d-7f1f-4e6c-8f03-61e7f940ac82 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:39.608150 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7c7dbc-9904-493b-a76e-428dc91a8e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:39.609009 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c7dbc-9904-493b-a76e-428dc91a8e83 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:39.609294 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c7dbc-9904-493b-a76e-428dc91a8e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:39.609573 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c7dbc-9904-493b-a76e-428dc91a8e83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:39.621183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7c7dbc-9904-493b-a76e-428dc91a8e83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:32:39.629155 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-5a7c7dbc-9904-493b-a76e-428dc91a8e83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:32:39.629155 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:39.629155 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:39.629155 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:39.629155 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:39.629155 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:39.629155 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:39.629155 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:39.629155 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:39.629155 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:39.629155 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:39.629155 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:39.629155 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:32:39.629155 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:32:39.629155 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:32:39.629155 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:32:39.630505 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:32:39.630505 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:32:39.630505 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:32:39.630505 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:32:39.630505 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:39.630505 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:39.630505 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:39.630505 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:39.630505 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:39.630505 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:39.630505 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:39.630505 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:39.630505 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:39.630505 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:39.630505 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:39.630505 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:39.631689 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:39.631689 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:39.631689 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:39.631689 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:39.631689 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:32:39.631689 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:32:39.631689 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:32:39.631689 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:32:39.631689 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:32:39.631689 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:32:39.631689 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:32:39.631689 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:32:39.631689 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 620/1249] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:32:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:39.636536 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8857eb98-b185-4af4-94ee-d41f67a343ca None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:39.751486 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8857eb98-b185-4af4-94ee-d41f67a343ca None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:39.788513 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:39.795038 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98abd4b8-7ef0-42eb-8b19-548640813cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:39.795760 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98abd4b8-7ef0-42eb-8b19-548640813cd1 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:39.796091 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98abd4b8-7ef0-42eb-8b19-548640813cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:39.796399 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98abd4b8-7ef0-42eb-8b19-548640813cd1 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:39.797216 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98abd4b8-7ef0-42eb-8b19-548640813cd1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:39.797527 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98abd4b8-7ef0-42eb-8b19-548640813cd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:39.799087 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98abd4b8-7ef0-42eb-8b19-548640813cd1 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:39.809136 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 630/1250] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:32:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:39.815178 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7c7dbc-9904-493b-a76e-428dc91a8e83 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:39.906912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:39.933675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7c7dbc-9904-493b-a76e-428dc91a8e83 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:39.972483 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f447500-64d1-4e77-8751-90d277ee23a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:39.973444 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f447500-64d1-4e77-8751-90d277ee23a4 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:39.973733 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f447500-64d1-4e77-8751-90d277ee23a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:39.974010 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f447500-64d1-4e77-8751-90d277ee23a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:39.985944 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f447500-64d1-4e77-8751-90d277ee23a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:39.987682 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f447500-64d1-4e77-8751-90d277ee23a4 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:40.005467 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-5f447500-64d1-4e77-8751-90d277ee23a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:40.005467 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:40.005467 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:40.005467 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:40.005467 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:40.005467 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:40.005467 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:40.005467 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:40.005467 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:40.005467 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:40.005467 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:40.005467 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:40.005467 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:40.005467 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:40.005467 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:40.006507 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:40.006507 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:40.006507 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:40.006507 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:40.006507 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:40.006507 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:40.006507 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:40.006507 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:40.006507 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:40.006507 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:40.006507 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:40.006507 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:40.006507 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:40.006507 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:40.006507 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:40.006507 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:40.006507 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:40.007649 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:40.007649 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:40.007649 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:40.007649 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:32:40.009194 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 621/1251] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:32:40.015137 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98abd4b8-7ef0-42eb-8b19-548640813cd1 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:40.129394 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98abd4b8-7ef0-42eb-8b19-548640813cd1 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:40.167937 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ade54-186f-4ded-b96b-e3291f139577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:40.168596 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ade54-186f-4ded-b96b-e3291f139577 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:40.168958 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ade54-186f-4ded-b96b-e3291f139577 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:40.169241 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ade54-186f-4ded-b96b-e3291f139577 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:40.170530 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ade54-186f-4ded-b96b-e3291f139577 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:40.170844 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ade54-186f-4ded-b96b-e3291f139577 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:40.172265 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ade54-186f-4ded-b96b-e3291f139577 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:40.185953 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 631/1252] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:40.192192 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f447500-64d1-4e77-8751-90d277ee23a4 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:40.308924 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f447500-64d1-4e77-8751-90d277ee23a4 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:40.336039 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:40.345736 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d28641-0916-4be7-b22c-d5c0392fa7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:40.346643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d28641-0916-4be7-b22c-d5c0392fa7af None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:40.346931 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d28641-0916-4be7-b22c-d5c0392fa7af None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:40.347241 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d28641-0916-4be7-b22c-d5c0392fa7af None admin] PATH_INFO: `/v3/users/placement` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:40.355619 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d28641-0916-4be7-b22c-d5c0392fa7af None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:40.357841 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d28641-0916-4be7-b22c-d5c0392fa7af None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:40.374068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-71d28641-0916-4be7-b22c-d5c0392fa7af None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:32:40.374068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:40.374068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:40.374068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:40.374068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:40.374068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:40.374068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:40.374068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:40.374068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:40.374068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:40.374068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:40.374068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:40.374068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:32:40.374068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:32:40.374068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:32:40.374068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:32:40.375327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:40.375327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:40.375327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:32:40.375327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:40.375327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:32:40.375327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:40.375327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:40.375327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:40.375327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:40.375327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:40.375327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:40.375327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:40.375327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:40.375327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:40.375327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:40.375327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:40.375327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:40.375327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:40.376362 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:40.376362 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:40.376362 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:32:40.376362 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:32:40.376362 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:32:40.376362 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:32:40.376362 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:32:40.376362 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:32:40.376362 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:32:40.376362 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:32:40.376362 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 622/1253] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:40.382127 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ade54-186f-4ded-b96b-e3291f139577 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:40.447801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:40.502864 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ade54-186f-4ded-b96b-e3291f139577 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:40.542020 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5aaaa7-6aa7-468e-8ad6-ecf69a96e21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:40.542739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5aaaa7-6aa7-468e-8ad6-ecf69a96e21b None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:40.543094 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5aaaa7-6aa7-468e-8ad6-ecf69a96e21b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:40.543374 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5aaaa7-6aa7-468e-8ad6-ecf69a96e21b None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:40.544296 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5aaaa7-6aa7-468e-8ad6-ecf69a96e21b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:40.544630 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5aaaa7-6aa7-468e-8ad6-ecf69a96e21b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:40.546059 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5aaaa7-6aa7-468e-8ad6-ecf69a96e21b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:40.564746 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 632/1254] 10.222.0.45 () {56 vars in 1143 bytes} [Wed Jun 15 14:32:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:40.570344 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d28641-0916-4be7-b22c-d5c0392fa7af None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:40.685283 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d28641-0916-4be7-b22c-d5c0392fa7af None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:40.724967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e8b2a-9e57-4209-82e0-8cc579eb0d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:40.725643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e8b2a-9e57-4209-82e0-8cc579eb0d7f None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:40.725928 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e8b2a-9e57-4209-82e0-8cc579eb0d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:40.726201 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e8b2a-9e57-4209-82e0-8cc579eb0d7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:40.801128 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:40.815440 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762e8b2a-9e57-4209-82e0-8cc579eb0d7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:40.816906 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762e8b2a-9e57-4209-82e0-8cc579eb0d7f None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:40.925929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 623/1255] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:32:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:32:40.931779 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5aaaa7-6aa7-468e-8ad6-ecf69a96e21b None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:41.034792 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5aaaa7-6aa7-468e-8ad6-ecf69a96e21b None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:41.048436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:41.070264 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3873fcd-0d36-42f7-97e1-1d312165eec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:41.071114 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3873fcd-0d36-42f7-97e1-1d312165eec6 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:41.071365 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3873fcd-0d36-42f7-97e1-1d312165eec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:41.071640 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3873fcd-0d36-42f7-97e1-1d312165eec6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:41.082694 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3873fcd-0d36-42f7-97e1-1d312165eec6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:41.084214 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3873fcd-0d36-42f7-97e1-1d312165eec6 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:41.101850 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-f3873fcd-0d36-42f7-97e1-1d312165eec6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:41.101850 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:41.101850 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:41.101850 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:41.101850 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:41.101850 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:41.101850 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:41.101850 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:41.101850 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:41.101850 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:41.101850 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:41.101850 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:41.101850 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:41.101850 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:41.101850 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:41.102808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:41.102808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:41.102808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:41.102808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:41.102808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:41.102808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:41.102808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:41.102808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:41.102808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:41.102808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:41.102808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:41.102808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:41.102808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:41.102808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:41.102808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:41.102808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:41.102808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:41.108502 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:41.108502 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:41.108502 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:41.108502 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:32:41.108502 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 633/1256] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:41.109849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e8b2a-9e57-4209-82e0-8cc579eb0d7f None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:41.213491 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:41.226967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e8b2a-9e57-4209-82e0-8cc579eb0d7f None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:41.263192 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58bf57-31f8-4706-b917-052869f69dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:41.263981 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58bf57-31f8-4706-b917-052869f69dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:41.264269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58bf57-31f8-4706-b917-052869f69dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:41.264542 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58bf57-31f8-4706-b917-052869f69dc3 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:41.265261 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f58bf57-31f8-4706-b917-052869f69dc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:41.265571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f58bf57-31f8-4706-b917-052869f69dc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:41.266935 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f58bf57-31f8-4706-b917-052869f69dc3 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:41.279490 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 624/1257] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:41.285271 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3873fcd-0d36-42f7-97e1-1d312165eec6 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:41.388679 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3873fcd-0d36-42f7-97e1-1d312165eec6 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:41.426045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce7f166-34f9-4b45-8464-74b2ac6c6fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:41.426837 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce7f166-34f9-4b45-8464-74b2ac6c6fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:41.427160 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce7f166-34f9-4b45-8464-74b2ac6c6fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:41.427441 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce7f166-34f9-4b45-8464-74b2ac6c6fc8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:41.434553 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce7f166-34f9-4b45-8464-74b2ac6c6fc8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:41.436865 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce7f166-34f9-4b45-8464-74b2ac6c6fc8 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:41.452571 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-2ce7f166-34f9-4b45-8464-74b2ac6c6fc8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:41.452571 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:41.452571 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:41.452571 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:41.452571 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:41.452571 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:41.452571 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:41.452571 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:41.452571 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:41.452571 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:41.452571 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:41.452571 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:41.452571 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:32:41.452571 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:32:41.452571 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:32:41.453515 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:32:41.453515 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:41.453515 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:41.453515 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:41.453515 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:41.453515 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:41.453515 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:41.453515 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:41.453515 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:41.453515 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:41.453515 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:41.453515 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:41.453515 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:41.453515 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:41.453515 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:41.453515 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:41.453515 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:41.454389 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:32:41.454389 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:32:41.454389 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:32:41.454389 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:32:41.454389 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:32:41.454389 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:32:41.454389 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:41.454389 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:32:41.458727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 634/1258] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:32:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:41.459756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58bf57-31f8-4706-b917-052869f69dc3 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:41.585147 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58bf57-31f8-4706-b917-052869f69dc3 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:41.623739 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aedfd8-629b-4071-8087-28671eb916c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:41.624464 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aedfd8-629b-4071-8087-28671eb916c4 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:41.624801 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aedfd8-629b-4071-8087-28671eb916c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:41.625125 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aedfd8-629b-4071-8087-28671eb916c4 None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:41.626007 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aedfd8-629b-4071-8087-28671eb916c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:41.626336 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aedfd8-629b-4071-8087-28671eb916c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:41.627944 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aedfd8-629b-4071-8087-28671eb916c4 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:41.642571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 625/1259] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:32:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:41.648611 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce7f166-34f9-4b45-8464-74b2ac6c6fc8 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:41.677396 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:41.783580 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce7f166-34f9-4b45-8464-74b2ac6c6fc8 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:41.823964 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754b5b52-a1c9-4a40-9267-2ac57d0aebcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:41.824834 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b5b52-a1c9-4a40-9267-2ac57d0aebcc None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:41.825230 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b5b52-a1c9-4a40-9267-2ac57d0aebcc None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:41.825599 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b5b52-a1c9-4a40-9267-2ac57d0aebcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:41.826770 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754b5b52-a1c9-4a40-9267-2ac57d0aebcc None admin] RBAC: Adding query filter params (role.id=ed1cfa2fcf724a4d84f5acea100930c7, scope.project.id=d488c79cb1704e9496fc196d8aa6c104, user.id=f4128111a9aa450abb2146a5d9926dec) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:41.827339 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754b5b52-a1c9-4a40-9267-2ac57d0aebcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:41.830005 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754b5b52-a1c9-4a40-9267-2ac57d0aebcc None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:41.836568 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 635/1260] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:32:41 2022] GET /identity/v3/role_assignments?user.id=f4128111a9aa450abb2146a5d9926dec&scope.project.id=d488c79cb1704e9496fc196d8aa6c104&role.id=ed1cfa2fcf724a4d84f5acea100930c7 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:43.979195 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b62ccc-0fd2-44dc-b3ad-bae8da1e830c None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:43.979738 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b62ccc-0fd2-44dc-b3ad-bae8da1e830c None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:43.979738 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b62ccc-0fd2-44dc-b3ad-bae8da1e830c None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:43.981049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 626/1261] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:32:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:43.986432 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5989dbf-8e87-4d7a-8650-6976b896641f None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:43.986815 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5989dbf-8e87-4d7a-8650-6976b896641f None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:43.987131 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5989dbf-8e87-4d7a-8650-6976b896641f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:44.004823 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-d5989dbf-8e87-4d7a-8650-6976b896641f None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:44.006023 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d5989dbf-8e87-4d7a-8650-6976b896641f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:44.071910 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 636/1262] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:44.094732 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b62ccc-0fd2-44dc-b3ad-bae8da1e830c None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:44.105523 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:44.197617 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b62ccc-0fd2-44dc-b3ad-bae8da1e830c None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:44.231308 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf517f-d9ba-475c-9da2-b742080dbeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:44.232507 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf517f-d9ba-475c-9da2-b742080dbeb0 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:44.232786 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf517f-d9ba-475c-9da2-b742080dbeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:44.233061 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf517f-d9ba-475c-9da2-b742080dbeb0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:44.243941 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdf517f-d9ba-475c-9da2-b742080dbeb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:32:44.252078 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-dbdf517f-d9ba-475c-9da2-b742080dbeb0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:32:44.252078 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:44.252078 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:44.252078 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:44.252078 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:44.252078 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:44.252078 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:44.252078 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:44.252078 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:44.252078 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:44.252078 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:44.252078 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:44.252078 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:32:44.252078 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:32:44.252078 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:32:44.252078 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:32:44.253358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:32:44.253358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:32:44.253358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:32:44.253358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:32:44.253358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:44.253358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:44.253358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:44.253358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:44.253358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:44.253358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:44.253358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:44.253358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:44.253358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:44.253358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:44.253358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:44.253358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:44.254584 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:44.254584 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:44.254584 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:44.254584 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:44.254584 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:32:44.254584 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:32:44.254584 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:32:44.254584 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:32:44.254584 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:32:44.254584 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:32:44.254584 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:32:44.254584 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:32:44.254584 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 627/1263] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:32:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:44.260145 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5989dbf-8e87-4d7a-8650-6976b896641f None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:44.278482 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:44.362535 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5989dbf-8e87-4d7a-8650-6976b896641f None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:44.395981 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2cdd7c-3603-4789-80b8-cf6c871bb243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:44.396637 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2cdd7c-3603-4789-80b8-cf6c871bb243 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:44.396915 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2cdd7c-3603-4789-80b8-cf6c871bb243 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:44.397183 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2cdd7c-3603-4789-80b8-cf6c871bb243 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:44.397920 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2cdd7c-3603-4789-80b8-cf6c871bb243 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:44.398222 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2cdd7c-3603-4789-80b8-cf6c871bb243 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:44.399621 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2cdd7c-3603-4789-80b8-cf6c871bb243 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:44.408187 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 637/1264] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:32:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:44.413909 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf517f-d9ba-475c-9da2-b742080dbeb0 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:44.482414 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:44.528946 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf517f-d9ba-475c-9da2-b742080dbeb0 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:44.569211 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaccb86-c003-4d42-b223-400562910b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:44.570008 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaccb86-c003-4d42-b223-400562910b59 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:44.570286 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaccb86-c003-4d42-b223-400562910b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:44.570560 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaccb86-c003-4d42-b223-400562910b59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:44.582038 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaccb86-c003-4d42-b223-400562910b59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:44.583525 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaccb86-c003-4d42-b223-400562910b59 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:44.601646 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-beaccb86-c003-4d42-b223-400562910b59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:44.601646 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:44.601646 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:44.601646 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:44.601646 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:44.601646 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:44.601646 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:44.601646 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:44.601646 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:44.601646 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:44.601646 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:44.601646 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:44.601646 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:44.601646 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:44.601646 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:44.602784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:44.602784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:44.602784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:44.602784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:44.602784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:44.602784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:44.602784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:44.602784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:44.602784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:44.602784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:44.602784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:44.602784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:44.602784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:44.602784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:44.602784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:44.602784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:44.602784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:44.603816 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:44.603816 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:44.603816 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:44.603816 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:32:44.603816 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 628/1265] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:44.609255 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2cdd7c-3603-4789-80b8-cf6c871bb243 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:44.690640 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:44.710470 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2cdd7c-3603-4789-80b8-cf6c871bb243 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:44.744753 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2162ec0-0a0f-41b2-a9d5-100a6a130a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:44.745330 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2162ec0-0a0f-41b2-a9d5-100a6a130a4f None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:44.745616 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2162ec0-0a0f-41b2-a9d5-100a6a130a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:44.745898 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2162ec0-0a0f-41b2-a9d5-100a6a130a4f None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:44.746629 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2162ec0-0a0f-41b2-a9d5-100a6a130a4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:44.746938 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2162ec0-0a0f-41b2-a9d5-100a6a130a4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:44.748353 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2162ec0-0a0f-41b2-a9d5-100a6a130a4f None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:44.760593 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 638/1266] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:44.767025 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaccb86-c003-4d42-b223-400562910b59 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:44.866163 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaccb86-c003-4d42-b223-400562910b59 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:44.885377 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:44.899984 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8084f2b4-c912-448f-8624-9918e32be8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:44.900794 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084f2b4-c912-448f-8624-9918e32be8e8 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:44.901085 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084f2b4-c912-448f-8624-9918e32be8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:44.901365 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084f2b4-c912-448f-8624-9918e32be8e8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:44.910025 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8084f2b4-c912-448f-8624-9918e32be8e8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:44.911514 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8084f2b4-c912-448f-8624-9918e32be8e8 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:44.925669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-8084f2b4-c912-448f-8624-9918e32be8e8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:32:44.925669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:44.925669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:44.925669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:44.925669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:44.925669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:44.925669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:44.925669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:44.925669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:44.925669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:44.925669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:44.925669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:44.925669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:32:44.925669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:32:44.925669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:32:44.925669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:32:44.926797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:44.926797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:44.926797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:32:44.926797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:44.926797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:32:44.926797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:44.926797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:44.926797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:44.926797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:44.926797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:44.926797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:44.926797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:44.926797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:44.926797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:44.926797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:44.926797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:44.926797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:44.926797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:44.927856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:44.927856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:44.927856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:32:44.927856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:32:44.927856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:32:44.927856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:32:44.927856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:32:44.927856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:32:44.927856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:32:44.927856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:32:44.927856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 629/1267] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:32:44.932791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2162ec0-0a0f-41b2-a9d5-100a6a130a4f None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:45.030095 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2162ec0-0a0f-41b2-a9d5-100a6a130a4f None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:45.064754 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6d39a-a5dd-40aa-b0c4-6bf5cd9430be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:45.065459 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6d39a-a5dd-40aa-b0c4-6bf5cd9430be None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:45.065749 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6d39a-a5dd-40aa-b0c4-6bf5cd9430be None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:45.066029 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6d39a-a5dd-40aa-b0c4-6bf5cd9430be None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:45.066869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f6d39a-a5dd-40aa-b0c4-6bf5cd9430be None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:45.067211 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f6d39a-a5dd-40aa-b0c4-6bf5cd9430be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:45.068658 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f6d39a-a5dd-40aa-b0c4-6bf5cd9430be None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:45.078438 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:45.087396 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 639/1268] 10.222.0.45 () {56 vars in 1143 bytes} [Wed Jun 15 14:32:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:45.093628 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8084f2b4-c912-448f-8624-9918e32be8e8 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:45.192535 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8084f2b4-c912-448f-8624-9918e32be8e8 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:45.225954 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c813378-e692-48ab-b1e1-51d765a298ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:45.226735 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c813378-e692-48ab-b1e1-51d765a298ed None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:45.227038 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c813378-e692-48ab-b1e1-51d765a298ed None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:45.227316 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c813378-e692-48ab-b1e1-51d765a298ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:45.237637 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c813378-e692-48ab-b1e1-51d765a298ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:45.239105 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c813378-e692-48ab-b1e1-51d765a298ed None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:45.256535 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-7c813378-e692-48ab-b1e1-51d765a298ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:45.256535 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:45.256535 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:45.256535 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:45.256535 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:45.256535 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:45.256535 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:45.256535 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:45.256535 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:45.256535 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:45.256535 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:45.256535 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:45.256535 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:45.256535 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:45.256535 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:45.257574 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:45.257574 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:45.257574 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:45.257574 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:45.257574 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:45.257574 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:45.257574 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:45.257574 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:45.257574 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:45.257574 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:45.257574 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:45.257574 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:45.257574 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:45.257574 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:45.257574 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:45.257574 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:45.257574 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:45.258484 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:45.258484 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:45.258484 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:45.258484 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:32:45.258484 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 630/1269] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:45.263523 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6d39a-a5dd-40aa-b0c4-6bf5cd9430be None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:45.362259 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6d39a-a5dd-40aa-b0c4-6bf5cd9430be None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:45.396433 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df10d2-d746-4fbf-85ba-2e4fba5b3763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:45.397090 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df10d2-d746-4fbf-85ba-2e4fba5b3763 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:45.397375 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df10d2-d746-4fbf-85ba-2e4fba5b3763 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:45.397650 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df10d2-d746-4fbf-85ba-2e4fba5b3763 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:45.398373 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9df10d2-d746-4fbf-85ba-2e4fba5b3763 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:45.398690 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9df10d2-d746-4fbf-85ba-2e4fba5b3763 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:45.400108 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9df10d2-d746-4fbf-85ba-2e4fba5b3763 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:45.411629 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 640/1270] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:45.417251 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c813378-e692-48ab-b1e1-51d765a298ed None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:45.448865 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:45.524460 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c813378-e692-48ab-b1e1-51d765a298ed None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:45.558863 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b038e9a-e6c8-4165-a4b1-7e6a703ac772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:45.559692 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b038e9a-e6c8-4165-a4b1-7e6a703ac772 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:45.559979 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b038e9a-e6c8-4165-a4b1-7e6a703ac772 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:45.560259 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b038e9a-e6c8-4165-a4b1-7e6a703ac772 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:45.567632 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b038e9a-e6c8-4165-a4b1-7e6a703ac772 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:45.569084 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b038e9a-e6c8-4165-a4b1-7e6a703ac772 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:45.584610 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-9b038e9a-e6c8-4165-a4b1-7e6a703ac772 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:45.584610 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:45.584610 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:45.584610 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:45.584610 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:45.584610 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:45.584610 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:45.584610 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:45.584610 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:45.584610 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:45.584610 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:45.584610 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:45.584610 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:32:45.584610 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:32:45.584610 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:32:45.585736 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:32:45.585736 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:45.585736 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:45.585736 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:45.585736 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:45.585736 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:45.585736 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:45.585736 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:45.585736 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:45.585736 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:45.585736 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:45.585736 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:45.585736 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:45.585736 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:45.585736 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:45.585736 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:45.585736 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:45.586875 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:32:45.586875 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:32:45.586875 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:32:45.586875 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:32:45.586875 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:32:45.586875 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:32:45.586875 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:45.586875 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:32:45.586875 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 631/1271] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:32:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:45.591734 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df10d2-d746-4fbf-85ba-2e4fba5b3763 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:45.651209 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:45.695197 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df10d2-d746-4fbf-85ba-2e4fba5b3763 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:45.731610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c0db9-c221-4e28-be2e-4fb58ac53722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:45.732271 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c0db9-c221-4e28-be2e-4fb58ac53722 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:45.732551 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c0db9-c221-4e28-be2e-4fb58ac53722 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:45.732826 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c0db9-c221-4e28-be2e-4fb58ac53722 None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:45.734147 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883c0db9-c221-4e28-be2e-4fb58ac53722 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:45.734452 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883c0db9-c221-4e28-be2e-4fb58ac53722 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:45.735925 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883c0db9-c221-4e28-be2e-4fb58ac53722 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:45.748098 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 641/1272] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:32:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:45.754650 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b038e9a-e6c8-4165-a4b1-7e6a703ac772 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:45.847064 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:45.856132 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b038e9a-e6c8-4165-a4b1-7e6a703ac772 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:45.890593 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae17491-6eba-4c51-be3b-3032d1281896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:45.891534 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae17491-6eba-4c51-be3b-3032d1281896 None admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:45.891815 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae17491-6eba-4c51-be3b-3032d1281896 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:45.892086 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae17491-6eba-4c51-be3b-3032d1281896 None admin] PATH_INFO: `/v3/projects/d488c79cb1704e9496fc196d8aa6c104/users/f4128111a9aa450abb2146a5d9926dec/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:45.894011 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae17491-6eba-4c51-be3b-3032d1281896 None admin] RBAC: Authorizing `identity:create_grant(project_id=d488c79cb1704e9496fc196d8aa6c104, user_id=f4128111a9aa450abb2146a5d9926dec, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:45.895479 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae17491-6eba-4c51-be3b-3032d1281896 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:45.908527 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 632/1273] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Jun 15 14:32:45 2022] PUT /identity/v3/projects/d488c79cb1704e9496fc196d8aa6c104/users/f4128111a9aa450abb2146a5d9926dec/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:32:48.034714 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df305c-1ac8-4177-a855-4eca1fce837c None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:48.035237 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df305c-1ac8-4177-a855-4eca1fce837c None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:48.035237 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df305c-1ac8-4177-a855-4eca1fce837c None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:48.036727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 642/1274] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:48.042132 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf3dc5-6e63-480f-a596-336e0f4f27f0 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:48.042552 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf3dc5-6e63-480f-a596-336e0f4f27f0 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:48.042697 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf3dc5-6e63-480f-a596-336e0f4f27f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:48.059143 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-8fbf3dc5-6e63-480f-a596-336e0f4f27f0 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:48.060347 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-8fbf3dc5-6e63-480f-a596-336e0f4f27f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:48.127082 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 633/1275] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:48.150687 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3df305c-1ac8-4177-a855-4eca1fce837c None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:48.254531 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3df305c-1ac8-4177-a855-4eca1fce837c None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:48.275643 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:48.293966 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa01b5-afce-40d4-9d3b-224aed09a830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:48.294765 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa01b5-afce-40d4-9d3b-224aed09a830 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:48.295103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa01b5-afce-40d4-9d3b-224aed09a830 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:48.295355 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa01b5-afce-40d4-9d3b-224aed09a830 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:48.306433 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa01b5-afce-40d4-9d3b-224aed09a830 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:32:48.314582 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-3baa01b5-afce-40d4-9d3b-224aed09a830 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:32:48.314582 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:48.314582 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:48.314582 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:48.314582 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:48.314582 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:48.314582 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:48.314582 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:48.314582 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:48.314582 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:48.314582 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:48.314582 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:48.314582 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:32:48.314582 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:32:48.314582 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:32:48.314582 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:32:48.315560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:32:48.315560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:32:48.315560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:32:48.315560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:32:48.315560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:48.315560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:48.315560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:48.315560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:48.315560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:48.315560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:48.315560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:48.315560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:48.315560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:48.315560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:48.315560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:48.315560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:48.316570 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:48.316570 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:48.316570 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:48.316570 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:48.316570 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:32:48.316570 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:32:48.316570 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:32:48.316570 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:32:48.316570 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:32:48.316570 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:32:48.316570 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:32:48.316570 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:32:48.316570 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 643/1276] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Jun 15 14:32:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:48.322846 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf3dc5-6e63-480f-a596-336e0f4f27f0 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:48.442240 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:48.462955 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf3dc5-6e63-480f-a596-336e0f4f27f0 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:48.504706 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1902470c-1361-49ae-a559-8054047acc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:48.505807 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902470c-1361-49ae-a559-8054047acc23 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:48.506102 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902470c-1361-49ae-a559-8054047acc23 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:48.506562 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902470c-1361-49ae-a559-8054047acc23 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:48.507638 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1902470c-1361-49ae-a559-8054047acc23 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:48.508060 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1902470c-1361-49ae-a559-8054047acc23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:48.509802 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1902470c-1361-49ae-a559-8054047acc23 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:48.522007 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 634/1277] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Jun 15 14:32:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:48.529196 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa01b5-afce-40d4-9d3b-224aed09a830 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:48.655998 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa01b5-afce-40d4-9d3b-224aed09a830 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:48.696295 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e37305-e84a-4a72-aa2c-764ef4e2f734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:48.697077 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e37305-e84a-4a72-aa2c-764ef4e2f734 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:48.697359 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e37305-e84a-4a72-aa2c-764ef4e2f734 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:48.697634 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e37305-e84a-4a72-aa2c-764ef4e2f734 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:48.708451 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e37305-e84a-4a72-aa2c-764ef4e2f734 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:48.709948 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e37305-e84a-4a72-aa2c-764ef4e2f734 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:48.739453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-e3e37305-e84a-4a72-aa2c-764ef4e2f734 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:48.739453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:48.739453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:48.739453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:48.739453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:48.739453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:48.739453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:48.739453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:48.739453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:48.739453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:48.739453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:48.739453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:48.739453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:48.739453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:48.739453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:48.740593 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:48.740593 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:48.740593 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:48.740593 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:48.740593 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:48.740593 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:48.740593 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:48.740593 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:48.740593 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:48.740593 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:48.740593 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:48.740593 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:48.740593 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:48.740593 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:48.740593 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:48.740593 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:48.740593 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:48.741683 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:48.741683 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:48.741683 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:48.741683 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:32:48.742022 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 644/1278] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:48.747746 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1902470c-1361-49ae-a559-8054047acc23 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:48.865946 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1902470c-1361-49ae-a559-8054047acc23 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:48.903665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91526249-26b7-494a-8732-7c5f09927681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:48.904324 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91526249-26b7-494a-8732-7c5f09927681 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:48.904605 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91526249-26b7-494a-8732-7c5f09927681 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:48.904877 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91526249-26b7-494a-8732-7c5f09927681 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:48.905642 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91526249-26b7-494a-8732-7c5f09927681 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:48.905945 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91526249-26b7-494a-8732-7c5f09927681 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:48.907405 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91526249-26b7-494a-8732-7c5f09927681 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:48.919513 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 635/1279] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:48.925974 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e37305-e84a-4a72-aa2c-764ef4e2f734 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:48.956939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:49.028837 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e37305-e84a-4a72-aa2c-764ef4e2f734 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:49.071504 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023dc75-7567-4735-ba62-c82f52e7fe5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:49.072315 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023dc75-7567-4735-ba62-c82f52e7fe5e None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:49.072596 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023dc75-7567-4735-ba62-c82f52e7fe5e None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:49.072865 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023dc75-7567-4735-ba62-c82f52e7fe5e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:49.080818 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0023dc75-7567-4735-ba62-c82f52e7fe5e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:49.082270 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0023dc75-7567-4735-ba62-c82f52e7fe5e None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:49.097143 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-0023dc75-7567-4735-ba62-c82f52e7fe5e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:32:49.097143 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:49.097143 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:49.097143 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:49.097143 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:49.097143 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:49.097143 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:49.097143 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:49.097143 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:49.097143 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:49.097143 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:49.097143 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:49.097143 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:32:49.097143 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:32:49.097143 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:32:49.097143 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:32:49.098302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:49.098302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:49.098302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:32:49.098302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:49.098302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:32:49.098302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:49.098302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:49.098302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:49.098302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:49.098302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:49.098302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:49.098302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:49.098302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:49.098302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:49.098302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:49.098302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:49.098302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:49.098302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:49.099312 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:49.099312 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:49.099312 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:32:49.099312 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:32:49.099312 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:32:49.099312 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:32:49.099312 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:32:49.099312 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:32:49.099312 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:32:49.099312 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:32:49.099312 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 645/1280] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:49.106253 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91526249-26b7-494a-8732-7c5f09927681 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:49.110831 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:49.209722 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91526249-26b7-494a-8732-7c5f09927681 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:49.246088 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5982126c-d271-49d5-8f84-93fe2579b55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:49.246756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982126c-d271-49d5-8f84-93fe2579b55a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:49.247070 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982126c-d271-49d5-8f84-93fe2579b55a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:49.247352 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982126c-d271-49d5-8f84-93fe2579b55a None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:49.248196 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5982126c-d271-49d5-8f84-93fe2579b55a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:49.248508 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5982126c-d271-49d5-8f84-93fe2579b55a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:49.249951 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5982126c-d271-49d5-8f84-93fe2579b55a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:49.268387 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 636/1281] 10.222.0.45 () {56 vars in 1143 bytes} [Wed Jun 15 14:32:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:49.274878 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023dc75-7567-4735-ba62-c82f52e7fe5e None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:49.385216 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:49.400707 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023dc75-7567-4735-ba62-c82f52e7fe5e None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:49.440264 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73350b83-007f-44bf-b117-06dab40d6b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:49.440938 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73350b83-007f-44bf-b117-06dab40d6b2c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:49.441216 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73350b83-007f-44bf-b117-06dab40d6b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:49.441494 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73350b83-007f-44bf-b117-06dab40d6b2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:49.523966 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73350b83-007f-44bf-b117-06dab40d6b2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:49.525356 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73350b83-007f-44bf-b117-06dab40d6b2c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:49.619699 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:49.630714 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 646/1282] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Jun 15 14:32:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:32:49.636895 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5982126c-d271-49d5-8f84-93fe2579b55a None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:49.701830 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:49.749531 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5982126c-d271-49d5-8f84-93fe2579b55a None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:49.784513 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2192ddd5-86d5-4656-8d64-f388e652e37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:49.785225 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192ddd5-86d5-4656-8d64-f388e652e37a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:49.785514 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192ddd5-86d5-4656-8d64-f388e652e37a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:49.785793 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192ddd5-86d5-4656-8d64-f388e652e37a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:49.795815 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2192ddd5-86d5-4656-8d64-f388e652e37a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:49.797387 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2192ddd5-86d5-4656-8d64-f388e652e37a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:49.815564 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-2192ddd5-86d5-4656-8d64-f388e652e37a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:49.815564 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:49.815564 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:49.815564 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:49.815564 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:49.815564 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:49.815564 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:49.815564 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:49.815564 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:49.815564 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:49.815564 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:49.815564 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:49.815564 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:49.815564 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:49.815564 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:49.816680 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:49.816680 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:49.816680 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:49.816680 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:49.816680 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:49.816680 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:49.816680 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:49.816680 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:49.816680 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:49.816680 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:49.816680 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:49.816680 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:49.816680 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:49.816680 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:49.816680 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:49.816680 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:49.816680 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:49.818007 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:49.818007 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:49.818007 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:49.818007 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:32:49.818007 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 637/1283] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:49.822542 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73350b83-007f-44bf-b117-06dab40d6b2c None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:49.923365 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73350b83-007f-44bf-b117-06dab40d6b2c None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:49.957774 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c2914-b93e-4ea4-ab55-2e1b5b7ef889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:49.958438 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c2914-b93e-4ea4-ab55-2e1b5b7ef889 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:49.958725 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c2914-b93e-4ea4-ab55-2e1b5b7ef889 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:49.959026 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c2914-b93e-4ea4-ab55-2e1b5b7ef889 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:49.959749 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0c2914-b93e-4ea4-ab55-2e1b5b7ef889 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:49.960062 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0c2914-b93e-4ea4-ab55-2e1b5b7ef889 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:49.961441 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0c2914-b93e-4ea4-ab55-2e1b5b7ef889 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:49.972579 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 647/1284] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:49.978369 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2192ddd5-86d5-4656-8d64-f388e652e37a None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:50.076115 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2192ddd5-86d5-4656-8d64-f388e652e37a None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:50.090470 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:50.111307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea899c6-ea4a-47c6-ac69-98a7756df308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:50.112105 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea899c6-ea4a-47c6-ac69-98a7756df308 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:50.112388 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea899c6-ea4a-47c6-ac69-98a7756df308 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:50.112664 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea899c6-ea4a-47c6-ac69-98a7756df308 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:50.119341 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea899c6-ea4a-47c6-ac69-98a7756df308 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:50.120786 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea899c6-ea4a-47c6-ac69-98a7756df308 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:50.134719 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-1ea899c6-ea4a-47c6-ac69-98a7756df308 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:50.134719 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:50.134719 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:50.134719 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:50.134719 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:50.134719 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:50.134719 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:50.134719 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:50.134719 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:50.134719 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:50.134719 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:50.134719 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:50.134719 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:32:50.134719 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:32:50.134719 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:32:50.135824 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:32:50.135824 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:50.135824 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:50.135824 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:50.135824 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:50.135824 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:50.135824 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:50.135824 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:50.135824 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:50.135824 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:50.135824 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:50.135824 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:50.135824 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:50.135824 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:50.135824 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:50.135824 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:50.135824 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:50.137161 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:32:50.137161 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:32:50.137161 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:32:50.137161 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:32:50.137161 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:32:50.137161 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:32:50.137161 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:50.137161 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Jun 15 14:32:50.137161 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 638/1285] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Jun 15 14:32:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:50.141874 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c2914-b93e-4ea4-ab55-2e1b5b7ef889 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:50.193615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:50.248547 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c2914-b93e-4ea4-ab55-2e1b5b7ef889 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:50.283928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb627904-598d-44c3-99b1-7d64b0d9ef5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:50.284593 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb627904-598d-44c3-99b1-7d64b0d9ef5c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:50.284880 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb627904-598d-44c3-99b1-7d64b0d9ef5c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:50.285155 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb627904-598d-44c3-99b1-7d64b0d9ef5c None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:50.285952 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb627904-598d-44c3-99b1-7d64b0d9ef5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:50.286264 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb627904-598d-44c3-99b1-7d64b0d9ef5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:50.287743 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb627904-598d-44c3-99b1-7d64b0d9ef5c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:50.300952 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 648/1286] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Jun 15 14:32:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:50.308126 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea899c6-ea4a-47c6-ac69-98a7756df308 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:50.411213 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea899c6-ea4a-47c6-ac69-98a7756df308 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:50.448487 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dab6f32-2515-4bf2-9cc5-73abcb635475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:50.449446 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab6f32-2515-4bf2-9cc5-73abcb635475 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:50.449879 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab6f32-2515-4bf2-9cc5-73abcb635475 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:50.450289 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab6f32-2515-4bf2-9cc5-73abcb635475 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:50.451627 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dab6f32-2515-4bf2-9cc5-73abcb635475 None admin] RBAC: Adding query filter params (role.id=ed1cfa2fcf724a4d84f5acea100930c7, scope.project.id=d488c79cb1704e9496fc196d8aa6c104, user.id=f4128111a9aa450abb2146a5d9926dec) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:50.452102 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dab6f32-2515-4bf2-9cc5-73abcb635475 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:50.454339 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dab6f32-2515-4bf2-9cc5-73abcb635475 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:50.462932 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 639/1287] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Jun 15 14:32:50 2022] GET /identity/v3/role_assignments?user.id=f4128111a9aa450abb2146a5d9926dec&scope.project.id=d488c79cb1704e9496fc196d8aa6c104&role.id=ed1cfa2fcf724a4d84f5acea100930c7 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:52.609384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09cf03c-62d4-47aa-b210-517994e36a98 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:52.609950 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09cf03c-62d4-47aa-b210-517994e36a98 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:52.609950 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09cf03c-62d4-47aa-b210-517994e36a98 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:52.611345 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 649/1288] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:32:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:52.616818 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7fc71-3d3c-4062-83ac-f5acaae9ca05 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:52.617490 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7fc71-3d3c-4062-83ac-f5acaae9ca05 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:52.617490 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7fc71-3d3c-4062-83ac-f5acaae9ca05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:52.634854 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-6ae7fc71-3d3c-4062-83ac-f5acaae9ca05 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:52.636092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-6ae7fc71-3d3c-4062-83ac-f5acaae9ca05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:52.666983 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:52.707579 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 640/1289] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:52.731555 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09cf03c-62d4-47aa-b210-517994e36a98 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:52.834125 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09cf03c-62d4-47aa-b210-517994e36a98 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:52.845006 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:52.873700 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6778803-9569-4ca5-a2cc-7de9317c4b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:52.874927 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6778803-9569-4ca5-a2cc-7de9317c4b6b None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:52.875248 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6778803-9569-4ca5-a2cc-7de9317c4b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:52.875525 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6778803-9569-4ca5-a2cc-7de9317c4b6b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:52.876177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6778803-9569-4ca5-a2cc-7de9317c4b6b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:52.877555 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6778803-9569-4ca5-a2cc-7de9317c4b6b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:52.889325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-d6778803-9569-4ca5-a2cc-7de9317c4b6b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:32:52.889325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:52.889325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:32:52.889325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:32:52.889325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:32:52.889325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:32:52.889325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:32:52.889325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:52.889325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:32:52.889325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:32:52.889325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:32:52.889325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:32:52.889325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:52.889325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:52.889325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:52.889325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:52.889325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:52.891146 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:52.891146 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:52.891146 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:52.891146 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:52.891146 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:52.891146 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:52.891146 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:32:52.891146 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:32:52.891146 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:32:52.891146 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:32:52.891146 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:52.891146 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:52.891146 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:52.891146 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:52.891146 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:52.891146 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:52.892209 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:52.892209 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:52.892209 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:52.892209 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:52.892209 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:52.892209 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:52.892209 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:52.892209 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:52.892209 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:52.892209 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:52.892209 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:32:52.892209 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:52.892209 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:32:52.892209 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:32:52.892209 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 650/1290] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Jun 15 14:32:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:52.897476 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae7fc71-3d3c-4062-83ac-f5acaae9ca05 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:52.997114 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae7fc71-3d3c-4062-83ac-f5acaae9ca05 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:53.031794 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762ba25-7765-45f6-9ac0-a5b9396bc6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:53.032459 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762ba25-7765-45f6-9ac0-a5b9396bc6f3 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:53.032742 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762ba25-7765-45f6-9ac0-a5b9396bc6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:53.033018 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762ba25-7765-45f6-9ac0-a5b9396bc6f3 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:53.033720 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762ba25-7765-45f6-9ac0-a5b9396bc6f3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:53.034026 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762ba25-7765-45f6-9ac0-a5b9396bc6f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:53.035424 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762ba25-7765-45f6-9ac0-a5b9396bc6f3 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:53.040396 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 641/1291] 10.222.0.45 () {56 vars in 1116 bytes} [Wed Jun 15 14:32:52 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:53.046294 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6778803-9569-4ca5-a2cc-7de9317c4b6b None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:53.144871 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6778803-9569-4ca5-a2cc-7de9317c4b6b None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:53.178695 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d168aa55-dcd7-4802-8e54-b850372af75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:53.179390 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168aa55-dcd7-4802-8e54-b850372af75d None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:53.179669 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168aa55-dcd7-4802-8e54-b850372af75d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:53.179940 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168aa55-dcd7-4802-8e54-b850372af75d None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:53.180644 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d168aa55-dcd7-4802-8e54-b850372af75d None admin] RBAC: Adding query filter params (type=placement) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:53.180944 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d168aa55-dcd7-4802-8e54-b850372af75d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:53.182304 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d168aa55-dcd7-4802-8e54-b850372af75d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:53.187772 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 651/1292] 10.222.0.45 () {56 vars in 1116 bytes} [Wed Jun 15 14:32:53 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:55.358397 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ba45d-5153-4c89-a749-82c0b7c3922f None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:55.358917 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ba45d-5153-4c89-a749-82c0b7c3922f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:55.359050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ba45d-5153-4c89-a749-82c0b7c3922f None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:55.360375 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 642/1293] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:55.367059 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af4c83-439a-478b-8f66-7e08bcf47866 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:55.367577 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af4c83-439a-478b-8f66-7e08bcf47866 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:55.368075 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af4c83-439a-478b-8f66-7e08bcf47866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:55.387427 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-e4af4c83-439a-478b-8f66-7e08bcf47866 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:55.388635 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e4af4c83-439a-478b-8f66-7e08bcf47866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:55.420645 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:55.459635 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 652/1294] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:55.483111 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ba45d-5153-4c89-a749-82c0b7c3922f None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:55.522377 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:55.598635 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ba45d-5153-4c89-a749-82c0b7c3922f None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:55.631762 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3706ba-1746-41c1-97cc-de66abe9f1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:55.632330 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3706ba-1746-41c1-97cc-de66abe9f1f5 None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:55.632609 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3706ba-1746-41c1-97cc-de66abe9f1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:55.632882 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3706ba-1746-41c1-97cc-de66abe9f1f5 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:55.633598 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3706ba-1746-41c1-97cc-de66abe9f1f5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:55.634955 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3706ba-1746-41c1-97cc-de66abe9f1f5 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:55.649008 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 643/1295] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Jun 15 14:32:55 2022] POST /identity/v3/services => generated 246 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:32:57.806184 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c972cd-31de-4638-ae8b-765bdc1cfb01 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:57.806720 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c972cd-31de-4638-ae8b-765bdc1cfb01 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:57.806888 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c972cd-31de-4638-ae8b-765bdc1cfb01 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:57.808196 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 653/1296] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:32:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:57.814433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dcbbfc-bcfa-453b-b7af-a68ac56c516d None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:57.814986 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dcbbfc-bcfa-453b-b7af-a68ac56c516d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:57.814986 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dcbbfc-bcfa-453b-b7af-a68ac56c516d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:57.832884 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-19dcbbfc-bcfa-453b-b7af-a68ac56c516d None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:57.834104 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-19dcbbfc-bcfa-453b-b7af-a68ac56c516d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:57.914317 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 644/1297] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:57 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:32:57.941877 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c972cd-31de-4638-ae8b-765bdc1cfb01 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:58.048851 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c972cd-31de-4638-ae8b-765bdc1cfb01 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:58.085337 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12be9137-24d9-4df0-8009-3d669e36078f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:58.086148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be9137-24d9-4df0-8009-3d669e36078f None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:58.086431 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be9137-24d9-4df0-8009-3d669e36078f None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:58.086705 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be9137-24d9-4df0-8009-3d669e36078f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:58.087400 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12be9137-24d9-4df0-8009-3d669e36078f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:58.088771 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12be9137-24d9-4df0-8009-3d669e36078f None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:58.101409 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-12be9137-24d9-4df0-8009-3d669e36078f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:32:58.101409 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:58.101409 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:32:58.101409 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:32:58.101409 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:32:58.101409 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:32:58.101409 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:32:58.101409 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:58.101409 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:32:58.101409 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:32:58.101409 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:32:58.101409 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:32:58.101409 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:58.101409 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:58.101409 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:58.101409 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:58.101409 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:58.102697 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:58.102697 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:58.102697 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:58.102697 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:58.102697 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:58.102697 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:58.102697 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:32:58.102697 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:32:58.102697 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:32:58.102697 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:32:58.102697 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:58.102697 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:58.102697 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:58.102697 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:58.102697 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:58.102697 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:58.103869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:58.103869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:58.103869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:58.103869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:58.103869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:58.103869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:58.103869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:58.103869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:58.103869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:58.103869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:58.103869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:32:58.103869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:58.103869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:32:58.103869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:32:58.103869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 654/1298] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Jun 15 14:32:57 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:58.109947 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dcbbfc-bcfa-453b-b7af-a68ac56c516d None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:58.130004 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:58.214964 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dcbbfc-bcfa-453b-b7af-a68ac56c516d None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:58.249842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13241ec1-73a0-45ca-8ff1-f011dacf1ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:58.250502 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13241ec1-73a0-45ca-8ff1-f011dacf1ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:58.250784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13241ec1-73a0-45ca-8ff1-f011dacf1ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:58.251118 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13241ec1-73a0-45ca-8ff1-f011dacf1ee2 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:58.251935 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13241ec1-73a0-45ca-8ff1-f011dacf1ee2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:58.252237 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13241ec1-73a0-45ca-8ff1-f011dacf1ee2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:58.253610 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13241ec1-73a0-45ca-8ff1-f011dacf1ee2 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:58.258932 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 645/1299] 10.222.0.45 () {56 vars in 1116 bytes} [Wed Jun 15 14:32:58 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:58.265044 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12be9137-24d9-4df0-8009-3d669e36078f None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:58.298270 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:58.409469 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12be9137-24d9-4df0-8009-3d669e36078f None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:58.464677 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767b962a-35f1-468c-9cfb-e0784122cab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:58.465377 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b962a-35f1-468c-9cfb-e0784122cab3 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:58.465658 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b962a-35f1-468c-9cfb-e0784122cab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:58.465929 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b962a-35f1-468c-9cfb-e0784122cab3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:58.466727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767b962a-35f1-468c-9cfb-e0784122cab3 None admin] RBAC: Adding query filter params (interface=public, service_id=585acb5d96f04ec2a373f814e3cc1049, region_id=RegionOne) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:58.467067 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767b962a-35f1-468c-9cfb-e0784122cab3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:58.469103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767b962a-35f1-468c-9cfb-e0784122cab3 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:58.474971 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 655/1300] 10.222.0.45 () {56 vars in 1251 bytes} [Wed Jun 15 14:32:58 2022] GET /identity/v3/endpoints?service_id=585acb5d96f04ec2a373f814e3cc1049&interface=public®ion_id=RegionOne => generated 194 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:58.480821 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13241ec1-73a0-45ca-8ff1-f011dacf1ee2 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:58.577721 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:58.597215 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13241ec1-73a0-45ca-8ff1-f011dacf1ee2 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:58.632687 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f29aa4-39f2-4175-a963-91f701cbddd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:58.633352 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f29aa4-39f2-4175-a963-91f701cbddd8 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:58.633635 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f29aa4-39f2-4175-a963-91f701cbddd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:58.633911 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f29aa4-39f2-4175-a963-91f701cbddd8 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:58.634594 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f29aa4-39f2-4175-a963-91f701cbddd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:58.636014 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f29aa4-39f2-4175-a963-91f701cbddd8 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:58.643993 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 646/1301] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Jun 15 14:32:58 2022] GET /identity/v3/services => generated 2191 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:33:00.813421 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85c795-b521-4ea7-bd25-139ca6a90dad None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:00.813941 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85c795-b521-4ea7-bd25-139ca6a90dad None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:00.813941 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85c795-b521-4ea7-bd25-139ca6a90dad None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:00.815382 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 656/1302] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:33:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:33:00.820829 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2a7dc-f50a-4b58-88c4-df9c3ba93e1e None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:00.821268 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2a7dc-f50a-4b58-88c4-df9c3ba93e1e None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:00.821375 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2a7dc-f50a-4b58-88c4-df9c3ba93e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:00.839604 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-15b2a7dc-f50a-4b58-88c4-df9c3ba93e1e None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:33:00.840825 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-15b2a7dc-f50a-4b58-88c4-df9c3ba93e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:33:00.923970 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 647/1303] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:33:00 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:33:00.946719 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85c795-b521-4ea7-bd25-139ca6a90dad None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:01.050451 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:01.057660 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85c795-b521-4ea7-bd25-139ca6a90dad None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:01.090822 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed57ae2-4944-4423-9453-c47bf66ca1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:01.091635 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed57ae2-4944-4423-9453-c47bf66ca1f9 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:01.091916 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed57ae2-4944-4423-9453-c47bf66ca1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:01.092186 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed57ae2-4944-4423-9453-c47bf66ca1f9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:01.093372 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed57ae2-4944-4423-9453-c47bf66ca1f9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:01.094740 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed57ae2-4944-4423-9453-c47bf66ca1f9 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:01.106436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-4ed57ae2-4944-4423-9453-c47bf66ca1f9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:33:01.106436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:01.106436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:33:01.106436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:33:01.106436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:33:01.106436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:33:01.106436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:33:01.106436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:33:01.106436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:33:01.106436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:33:01.106436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:33:01.106436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:33:01.106436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:01.106436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:01.106436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:01.106436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:01.106436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:01.108248 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:01.108248 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:01.108248 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:01.108248 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:01.108248 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:01.108248 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:01.108248 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:33:01.108248 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:33:01.108248 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:33:01.108248 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:33:01.108248 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:01.108248 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:01.108248 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:01.108248 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:01.108248 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:01.108248 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:01.111215 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:01.111215 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:01.111215 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:01.111215 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:01.111215 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:01.111215 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:01.111215 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:01.111215 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:01.111215 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:01.111215 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:01.111215 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:33:01.111215 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:33:01.111215 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:33:01.111215 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Jun 15 14:33:01.111215 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 657/1304] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Jun 15 14:33:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:33:01.115409 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b2a7dc-f50a-4b58-88c4-df9c3ba93e1e None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:01.192850 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:01.215898 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b2a7dc-f50a-4b58-88c4-df9c3ba93e1e None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:01.251791 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c435687b-f470-43ad-8e85-2d00bbf18766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:01.252449 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435687b-f470-43ad-8e85-2d00bbf18766 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:01.252727 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435687b-f470-43ad-8e85-2d00bbf18766 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:01.252997 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435687b-f470-43ad-8e85-2d00bbf18766 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:01.253694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c435687b-f470-43ad-8e85-2d00bbf18766 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:01.253998 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c435687b-f470-43ad-8e85-2d00bbf18766 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:01.255393 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c435687b-f470-43ad-8e85-2d00bbf18766 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:01.260629 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 648/1305] 10.222.0.45 () {56 vars in 1116 bytes} [Wed Jun 15 14:33:01 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:01.266812 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed57ae2-4944-4423-9453-c47bf66ca1f9 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:01.378665 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed57ae2-4944-4423-9453-c47bf66ca1f9 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:01.428798 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea78120-b587-45bb-a5f5-bc8a9e0f6ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:01.429913 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea78120-b587-45bb-a5f5-bc8a9e0f6ac8 None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:01.430424 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea78120-b587-45bb-a5f5-bc8a9e0f6ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:01.430929 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea78120-b587-45bb-a5f5-bc8a9e0f6ac8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:01.432313 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea78120-b587-45bb-a5f5-bc8a9e0f6ac8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:01.434963 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea78120-b587-45bb-a5f5-bc8a9e0f6ac8 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:01.467313 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 658/1306] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Jun 15 14:33:01 2022] POST /identity/v3/endpoints => generated 331 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:33:11.259430 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=30 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:11.458185 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:08.407561 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608906a-5ba8-4c58-833c-0f956c73a6cc None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:08.408093 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608906a-5ba8-4c58-833c-0f956c73a6cc None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:08.408359 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608906a-5ba8-4c58-833c-0f956c73a6cc None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:08.409679 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 649/1307] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:08.414961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b93e1c8-342b-4383-8763-870b10716f35 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:08.415433 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b93e1c8-342b-4383-8763-870b10716f35 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:08.415636 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b93e1c8-342b-4383-8763-870b10716f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:08.516573 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-0b93e1c8-342b-4383-8763-870b10716f35 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:08.517785 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-0b93e1c8-342b-4383-8763-870b10716f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:08.671855 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 659/1308] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:34:08.720051 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f353049-0abc-4e74-b8ce-f1bb4062f5fb None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:08.720051 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f353049-0abc-4e74-b8ce-f1bb4062f5fb None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:08.720051 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f353049-0abc-4e74-b8ce-f1bb4062f5fb None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:08.722222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 650/1309] 10.222.0.45 () {54 vars in 964 bytes} [Wed Jun 15 14:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:34:08.735130 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568db89-1b3e-4176-8213-ee6610dca025 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:08.735587 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568db89-1b3e-4176-8213-ee6610dca025 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:08.736011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568db89-1b3e-4176-8213-ee6610dca025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:08.926293 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-c568db89-1b3e-4176-8213-ee6610dca025 None None] MFA Rules not processed for user `975ae2790ac849ff88e991231671f510`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:08.927568 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c568db89-1b3e-4176-8213-ee6610dca025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:08.980322 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:09.096423 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 660/1310] 10.222.0.45 () {58 vars in 1061 bytes} [Wed Jun 15 14:34:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:34:09.106674 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f353049-0abc-4e74-b8ce-f1bb4062f5fb None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:09.109355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-2f353049-0abc-4e74-b8ce-f1bb4062f5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:09.284764 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f353049-0abc-4e74-b8ce-f1bb4062f5fb None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:09.378424 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7079ed-1e8e-4182-80b2-d84a9ea9a2c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:09.379134 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7079ed-1e8e-4182-80b2-d84a9ea9a2c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:09.379422 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7079ed-1e8e-4182-80b2-d84a9ea9a2c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:09.379701 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7079ed-1e8e-4182-80b2-d84a9ea9a2c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:09.381564 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-3d7079ed-1e8e-4182-80b2-d84a9ea9a2c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:09.457261 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:09.540154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7079ed-1e8e-4182-80b2-d84a9ea9a2c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:09.541882 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7079ed-1e8e-4182-80b2-d84a9ea9a2c8 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:09.565617 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 651/1311] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:34:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:34:12.131209 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c966f3-0f27-4aef-b708-47c1fd63dc0d None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:12.131743 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c966f3-0f27-4aef-b708-47c1fd63dc0d None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:12.131743 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c966f3-0f27-4aef-b708-47c1fd63dc0d None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:12.133206 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 661/1312] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:12.138788 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e4b1f-7f02-4bf4-a000-bf24539a17d3 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:12.139450 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e4b1f-7f02-4bf4-a000-bf24539a17d3 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:12.139450 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e4b1f-7f02-4bf4-a000-bf24539a17d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:12.226786 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-e14e4b1f-7f02-4bf4-a000-bf24539a17d3 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:12.229899 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e14e4b1f-7f02-4bf4-a000-bf24539a17d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:12.332173 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 652/1313] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:34:12.389574 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d14c75-f1ac-4a96-9bda-03ba1c9d295f None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:12.389784 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d14c75-f1ac-4a96-9bda-03ba1c9d295f None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:12.390047 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d14c75-f1ac-4a96-9bda-03ba1c9d295f None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:12.391520 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 662/1314] 10.222.0.45 () {54 vars in 964 bytes} [Wed Jun 15 14:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:12.398282 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a666114-fff8-4276-8458-d23a95177dc3 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:12.398585 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a666114-fff8-4276-8458-d23a95177dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:12.398869 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a666114-fff8-4276-8458-d23a95177dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:12.410401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:12.488100 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-2a666114-fff8-4276-8458-d23a95177dc3 None None] MFA Rules not processed for user `975ae2790ac849ff88e991231671f510`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:12.489340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-2a666114-fff8-4276-8458-d23a95177dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:12.571705 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 653/1315] 10.222.0.45 () {58 vars in 1061 bytes} [Wed Jun 15 14:34:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:34:12.581355 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d14c75-f1ac-4a96-9bda-03ba1c9d295f None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:12.582896 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-59d14c75-f1ac-4a96-9bda-03ba1c9d295f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:12.693148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d14c75-f1ac-4a96-9bda-03ba1c9d295f None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:12.773153 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:12.776192 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b434d-7fea-493d-a718-1733d7f49f71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:12.777039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b434d-7fea-493d-a718-1733d7f49f71 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:12.777571 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b434d-7fea-493d-a718-1733d7f49f71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:12.777935 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b434d-7fea-493d-a718-1733d7f49f71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:12.779990 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-8b8b434d-7fea-493d-a718-1733d7f49f71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:12.915827 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8b434d-7fea-493d-a718-1733d7f49f71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:12.917280 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8b434d-7fea-493d-a718-1733d7f49f71 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:12.937026 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 663/1316] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:34:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:34:15.588090 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e9c2e-4321-4d8d-af1a-7df2c1b70dde None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:15.588685 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e9c2e-4321-4d8d-af1a-7df2c1b70dde None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:15.588685 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e9c2e-4321-4d8d-af1a-7df2c1b70dde None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:15.590016 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 654/1317] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:34:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:15.595376 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ae8b0-748e-47d7-a971-dd799d34fab0 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:15.595813 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ae8b0-748e-47d7-a971-dd799d34fab0 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:15.595939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ae8b0-748e-47d7-a971-dd799d34fab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:15.659218 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-f06ae8b0-748e-47d7-a971-dd799d34fab0 None None] MFA Rules not processed for user `34d8aa4f47ae452592b3481bdb02e32a`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:15.660411 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f06ae8b0-748e-47d7-a971-dd799d34fab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:15.714651 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:15.760330 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 664/1318] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:15 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:34:15.800468 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e9c2e-4321-4d8d-af1a-7df2c1b70dde None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:15.802012 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-9a0e9c2e-4321-4d8d-af1a-7df2c1b70dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:15.903655 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e9c2e-4321-4d8d-af1a-7df2c1b70dde None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:15.906380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:15.975923 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d767b7e8-4181-4237-a841-a9954b5e0a67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:15.976563 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d767b7e8-4181-4237-a841-a9954b5e0a67 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:15.976843 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d767b7e8-4181-4237-a841-a9954b5e0a67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:15.977116 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d767b7e8-4181-4237-a841-a9954b5e0a67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:15.978868 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d767b7e8-4181-4237-a841-a9954b5e0a67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:16.097519 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d767b7e8-4181-4237-a841-a9954b5e0a67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:16.099014 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d767b7e8-4181-4237-a841-a9954b5e0a67 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:16.117270 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 655/1319] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:34:15 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:34:18.932933 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7034b75-8b79-40fc-84a1-60e346ed1022 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:18.933478 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7034b75-8b79-40fc-84a1-60e346ed1022 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:18.933478 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7034b75-8b79-40fc-84a1-60e346ed1022 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:18.934860 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 665/1320] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:34:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:18.940664 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d2657-285a-4cab-a4fe-ca3c0832a5d6 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:18.941079 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d2657-285a-4cab-a4fe-ca3c0832a5d6 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:18.941221 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d2657-285a-4cab-a4fe-ca3c0832a5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:18.979964 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:19.005355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-149d2657-285a-4cab-a4fe-ca3c0832a5d6 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:19.006570 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-149d2657-285a-4cab-a4fe-ca3c0832a5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:19.100405 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 656/1321] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:18 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:34:19.139042 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7034b75-8b79-40fc-84a1-60e346ed1022 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:19.140585 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a7034b75-8b79-40fc-84a1-60e346ed1022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:19.243616 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7034b75-8b79-40fc-84a1-60e346ed1022 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:19.315553 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e2ce3-1fa7-4355-bccf-e903c595fa88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:19.316197 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e2ce3-1fa7-4355-bccf-e903c595fa88 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:19.316481 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e2ce3-1fa7-4355-bccf-e903c595fa88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:19.316758 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e2ce3-1fa7-4355-bccf-e903c595fa88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:19.318405 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-614e2ce3-1fa7-4355-bccf-e903c595fa88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:19.325728 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:19.429647 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614e2ce3-1fa7-4355-bccf-e903c595fa88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:19.431264 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614e2ce3-1fa7-4355-bccf-e903c595fa88 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:19.448916 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 666/1322] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:34:19 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:34:22.295237 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b0d85-022b-406b-aaee-e45ac166aa31 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:22.295784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b0d85-022b-406b-aaee-e45ac166aa31 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:22.295784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b0d85-022b-406b-aaee-e45ac166aa31 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:22.297173 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 657/1323] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:22.302472 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb55c8-eedf-45ed-a3f9-339ac8f84ccf None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:22.302856 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb55c8-eedf-45ed-a3f9-339ac8f84ccf None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:22.303181 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb55c8-eedf-45ed-a3f9-339ac8f84ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:22.381542 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-73fb55c8-eedf-45ed-a3f9-339ac8f84ccf None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:22.382775 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-73fb55c8-eedf-45ed-a3f9-339ac8f84ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:22.445886 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:22.488398 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 667/1324] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:34:22.524489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b0d85-022b-406b-aaee-e45ac166aa31 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:22.526069 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-227b0d85-022b-406b-aaee-e45ac166aa31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:22.632765 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b0d85-022b-406b-aaee-e45ac166aa31 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:22.645238 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:22.710289 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7599c155-fbb0-4722-a5f4-a45beb176acc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:22.710937 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599c155-fbb0-4722-a5f4-a45beb176acc service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:22.711248 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599c155-fbb0-4722-a5f4-a45beb176acc service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:22.711529 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599c155-fbb0-4722-a5f4-a45beb176acc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:22.713173 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-7599c155-fbb0-4722-a5f4-a45beb176acc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:22.828805 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7599c155-fbb0-4722-a5f4-a45beb176acc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:22.830257 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7599c155-fbb0-4722-a5f4-a45beb176acc service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:22.848761 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 658/1325] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:34:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:34:25.034489 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9259f9-9529-4d77-aab9-857537d80a4e None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:25.035059 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9259f9-9529-4d77-aab9-857537d80a4e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:25.035059 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9259f9-9529-4d77-aab9-857537d80a4e None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:25.036337 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 668/1326] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:25.041439 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747dd8e-f69d-451a-8cdf-676e0da89684 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:25.041854 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747dd8e-f69d-451a-8cdf-676e0da89684 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:25.041989 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747dd8e-f69d-451a-8cdf-676e0da89684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:25.088050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:25.107004 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-f747dd8e-f69d-451a-8cdf-676e0da89684 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:25.108763 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f747dd8e-f69d-451a-8cdf-676e0da89684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:25.203206 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 659/1327] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:34:25.239074 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9259f9-9529-4d77-aab9-857537d80a4e None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:25.240595 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ab9259f9-9529-4d77-aab9-857537d80a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:25.352602 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9259f9-9529-4d77-aab9-857537d80a4e None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:25.426220 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94168de-a2fd-4a5b-a5f3-1d0ee10db0f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:25.426868 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94168de-a2fd-4a5b-a5f3-1d0ee10db0f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:25.427185 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94168de-a2fd-4a5b-a5f3-1d0ee10db0f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:25.427465 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94168de-a2fd-4a5b-a5f3-1d0ee10db0f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:25.429124 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c94168de-a2fd-4a5b-a5f3-1d0ee10db0f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:25.446108 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:25.547652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94168de-a2fd-4a5b-a5f3-1d0ee10db0f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:25.549103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94168de-a2fd-4a5b-a5f3-1d0ee10db0f7 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:25.567437 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 669/1328] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:34:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:34:27.774412 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea5cc8-dbaa-4c50-af13-b1ad4ec469a4 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:27.774963 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea5cc8-dbaa-4c50-af13-b1ad4ec469a4 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:27.774963 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea5cc8-dbaa-4c50-af13-b1ad4ec469a4 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:27.776387 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 660/1329] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:34:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:27.781857 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbea5e7-2673-4548-9bfc-c5367a46bddd None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:27.782262 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbea5e7-2673-4548-9bfc-c5367a46bddd None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:27.782496 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbea5e7-2673-4548-9bfc-c5367a46bddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:27.848589 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-dcbea5e7-2673-4548-9bfc-c5367a46bddd None None] MFA Rules not processed for user `34d8aa4f47ae452592b3481bdb02e32a`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:27.849960 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-dcbea5e7-2673-4548-9bfc-c5367a46bddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:27.917672 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:27.950480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 670/1330] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:27 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:34:27.988138 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea5cc8-dbaa-4c50-af13-b1ad4ec469a4 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:27.989702 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-a8ea5cc8-dbaa-4c50-af13-b1ad4ec469a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:28.125340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea5cc8-dbaa-4c50-af13-b1ad4ec469a4 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:28.146027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:28.201397 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c5df93-ccf0-4223-9971-30899a0b0bf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:28.202046 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5df93-ccf0-4223-9971-30899a0b0bf4 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:28.202330 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5df93-ccf0-4223-9971-30899a0b0bf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:28.202602 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5df93-ccf0-4223-9971-30899a0b0bf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:28.204325 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-37c5df93-ccf0-4223-9971-30899a0b0bf4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:28.314386 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c5df93-ccf0-4223-9971-30899a0b0bf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:28.315883 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c5df93-ccf0-4223-9971-30899a0b0bf4 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:28.335766 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 661/1331] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:34:27 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:34:30.729707 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec80b90f-6099-4c38-afca-73e77f304bb6 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:30.730185 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec80b90f-6099-4c38-afca-73e77f304bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:30.730291 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec80b90f-6099-4c38-afca-73e77f304bb6 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:30.731665 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 671/1332] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:34:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:30.736722 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be2951-65da-4cfb-aaa8-55a1b3180abb None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:30.737118 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be2951-65da-4cfb-aaa8-55a1b3180abb None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:30.737268 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be2951-65da-4cfb-aaa8-55a1b3180abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:30.799088 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:30.801621 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-10be2951-65da-4cfb-aaa8-55a1b3180abb None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:30.803475 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-10be2951-65da-4cfb-aaa8-55a1b3180abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:30.902626 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 662/1333] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:34:30.940666 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec80b90f-6099-4c38-afca-73e77f304bb6 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:30.942245 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ec80b90f-6099-4c38-afca-73e77f304bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:31.046588 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec80b90f-6099-4c38-afca-73e77f304bb6 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:31.122571 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d0f25e-9aa1-4800-bff0-5310d3df4f8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:31.123244 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0f25e-9aa1-4800-bff0-5310d3df4f8c service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:31.123530 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0f25e-9aa1-4800-bff0-5310d3df4f8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:31.123805 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0f25e-9aa1-4800-bff0-5310d3df4f8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:31.125463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-01d0f25e-9aa1-4800-bff0-5310d3df4f8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:31.152545 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:31.235295 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d0f25e-9aa1-4800-bff0-5310d3df4f8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:31.236963 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d0f25e-9aa1-4800-bff0-5310d3df4f8c service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:31.257899 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 672/1334] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:34:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:34:33.480055 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dccc884-e6fa-4de7-a415-e64f02acf7fe None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:33.480587 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dccc884-e6fa-4de7-a415-e64f02acf7fe None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:33.481252 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dccc884-e6fa-4de7-a415-e64f02acf7fe None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:33.482603 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 663/1335] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:34:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:33.488296 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841da91-9916-421c-a359-7ee91eb575c9 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:33.488765 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841da91-9916-421c-a359-7ee91eb575c9 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:33.488879 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841da91-9916-421c-a359-7ee91eb575c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:33.552770 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-4841da91-9916-421c-a359-7ee91eb575c9 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:33.553966 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-4841da91-9916-421c-a359-7ee91eb575c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:33.627404 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:33.650199 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 673/1336] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:34:33.690767 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dccc884-e6fa-4de7-a415-e64f02acf7fe None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:33.692362 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-4dccc884-e6fa-4de7-a415-e64f02acf7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:33.801451 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dccc884-e6fa-4de7-a415-e64f02acf7fe None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:33.831902 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:33.876587 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a7385-4d5b-4148-b396-fc522f41b681 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:33.877229 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a7385-4d5b-4148-b396-fc522f41b681 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:33.877508 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a7385-4d5b-4148-b396-fc522f41b681 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:33.877780 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a7385-4d5b-4148-b396-fc522f41b681 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:33.879484 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-4a9a7385-4d5b-4148-b396-fc522f41b681 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:33.986464 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a7385-4d5b-4148-b396-fc522f41b681 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:33.987928 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a7385-4d5b-4148-b396-fc522f41b681 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:34.006255 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 664/1337] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:34:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:34:36.580200 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff11c85-fe10-42a2-8e20-5e36e901f950 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:36.580728 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff11c85-fe10-42a2-8e20-5e36e901f950 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:36.580845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff11c85-fe10-42a2-8e20-5e36e901f950 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:36.582179 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 674/1338] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:36.587534 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b8701-7829-4533-98f7-a6544919d5cf None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:36.587947 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b8701-7829-4533-98f7-a6544919d5cf None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:36.588093 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b8701-7829-4533-98f7-a6544919d5cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:36.664565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-803b8701-7829-4533-98f7-a6544919d5cf None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:36.665798 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-803b8701-7829-4533-98f7-a6544919d5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:36.675435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:36.766495 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 665/1339] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:36 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:34:36.802319 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff11c85-fe10-42a2-8e20-5e36e901f950 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:36.803895 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-cff11c85-fe10-42a2-8e20-5e36e901f950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:36.913352 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff11c85-fe10-42a2-8e20-5e36e901f950 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:36.993705 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d6dd03-4f7c-4c9f-91ed-9642a012b713 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:36.994351 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6dd03-4f7c-4c9f-91ed-9642a012b713 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:36.994636 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6dd03-4f7c-4c9f-91ed-9642a012b713 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:36.994912 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6dd03-4f7c-4c9f-91ed-9642a012b713 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:36.997133 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-99d6dd03-4f7c-4c9f-91ed-9642a012b713 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:37.032551 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:37.106197 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d6dd03-4f7c-4c9f-91ed-9642a012b713 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:37.107686 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d6dd03-4f7c-4c9f-91ed-9642a012b713 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:37.125962 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 675/1340] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:34:36 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:34:41.248550 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69792d3d-c614-4f69-8472-16e6e199ab7e None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:41.249096 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69792d3d-c614-4f69-8472-16e6e199ab7e None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:41.249096 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69792d3d-c614-4f69-8472-16e6e199ab7e None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:41.250508 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 666/1341] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:34:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:41.255885 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6a4bf4-d8f9-4dc3-86ce-6b5de485698d None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:41.256287 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6a4bf4-d8f9-4dc3-86ce-6b5de485698d None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:41.256441 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6a4bf4-d8f9-4dc3-86ce-6b5de485698d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:41.326976 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-fd6a4bf4-d8f9-4dc3-86ce-6b5de485698d None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:41.328216 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-fd6a4bf4-d8f9-4dc3-86ce-6b5de485698d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:41.419801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:41.434315 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 676/1342] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:34:41.477988 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69792d3d-c614-4f69-8472-16e6e199ab7e None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:41.480784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-69792d3d-c614-4f69-8472-16e6e199ab7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:41.582765 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69792d3d-c614-4f69-8472-16e6e199ab7e None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:41.586393 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe174474-797f-4da9-aba4-0514d4452fbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:41.587063 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe174474-797f-4da9-aba4-0514d4452fbb service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:41.587346 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe174474-797f-4da9-aba4-0514d4452fbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:41.587618 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe174474-797f-4da9-aba4-0514d4452fbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:41.597134 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe174474-797f-4da9-aba4-0514d4452fbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:41.598548 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe174474-797f-4da9-aba4-0514d4452fbb service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:41.608419 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 667/1343] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:34:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:34:44.393742 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cee71-2f89-4a3c-88ab-691b919e3c8c None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:44.394357 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cee71-2f89-4a3c-88ab-691b919e3c8c None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:44.394357 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cee71-2f89-4a3c-88ab-691b919e3c8c None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:44.395943 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 677/1344] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:34:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:44.401415 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3bf9e4-4098-471c-adb0-987116e77320 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:44.402796 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3bf9e4-4098-471c-adb0-987116e77320 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:44.402796 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3bf9e4-4098-471c-adb0-987116e77320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:44.431804 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-cb3bf9e4-4098-471c-adb0-987116e77320 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:44.433059 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-cb3bf9e4-4098-471c-adb0-987116e77320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:44.464069 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:44.520126 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 668/1345] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:44 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:34:44.564000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7cee71-2f89-4a3c-88ab-691b919e3c8c None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:44.576241 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7cee71-2f89-4a3c-88ab-691b919e3c8c None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:44.579818 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903fa099-a066-4ee2-afdd-2ad5d53586aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:44.580463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903fa099-a066-4ee2-afdd-2ad5d53586aa service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:44.580751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903fa099-a066-4ee2-afdd-2ad5d53586aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:44.581030 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903fa099-a066-4ee2-afdd-2ad5d53586aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:44.590461 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903fa099-a066-4ee2-afdd-2ad5d53586aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:44.592014 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903fa099-a066-4ee2-afdd-2ad5d53586aa service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:44.602175 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 678/1346] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:34:44 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:34:48.766740 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ffd50-9b72-481b-b4d2-693232a0e2c1 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:48.767361 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ffd50-9b72-481b-b4d2-693232a0e2c1 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:48.767361 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ffd50-9b72-481b-b4d2-693232a0e2c1 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:48.768819 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 669/1347] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:34:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:48.773987 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22f857-8920-4e5f-a381-6a676e2cdb70 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:48.774456 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22f857-8920-4e5f-a381-6a676e2cdb70 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:48.774557 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22f857-8920-4e5f-a381-6a676e2cdb70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:48.794787 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-cf22f857-8920-4e5f-a381-6a676e2cdb70 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:48.796021 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-cf22f857-8920-4e5f-a381-6a676e2cdb70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:48.803917 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 679/1348] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:34:48.841545 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5ffd50-9b72-481b-b4d2-693232a0e2c1 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:48.852628 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5ffd50-9b72-481b-b4d2-693232a0e2c1 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:48.856245 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d5068f-0fe3-4dfa-b08b-faab365af458 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:48.856878 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d5068f-0fe3-4dfa-b08b-faab365af458 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:48.857156 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d5068f-0fe3-4dfa-b08b-faab365af458 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:48.857430 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d5068f-0fe3-4dfa-b08b-faab365af458 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:48.866918 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d5068f-0fe3-4dfa-b08b-faab365af458 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:48.868721 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d5068f-0fe3-4dfa-b08b-faab365af458 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:48.878515 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 670/1349] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:34:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:34:53.960281 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ace77-a3b6-47d8-a270-ba5c9cf22293 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:53.960722 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ace77-a3b6-47d8-a270-ba5c9cf22293 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:53.960819 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ace77-a3b6-47d8-a270-ba5c9cf22293 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:53.962538 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 680/1350] 10.222.0.45 () {54 vars in 921 bytes} [Wed Jun 15 14:34:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:53.967414 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf60ba3-74fe-4bae-9ed5-6db4fc8d9e97 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:53.967811 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf60ba3-74fe-4bae-9ed5-6db4fc8d9e97 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:53.968099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf60ba3-74fe-4bae-9ed5-6db4fc8d9e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:54.132204 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-faf60ba3-74fe-4bae-9ed5-6db4fc8d9e97 None None] MFA Rules not processed for user `f4128111a9aa450abb2146a5d9926dec`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:54.134561 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-faf60ba3-74fe-4bae-9ed5-6db4fc8d9e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:54.282783 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 671/1351] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Jun 15 14:34:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:34:58.586330 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7968ef-60e5-40fc-bb96-f9dddb95de78 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:58.586935 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7968ef-60e5-40fc-bb96-f9dddb95de78 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:58.586935 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7968ef-60e5-40fc-bb96-f9dddb95de78 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:58.588501 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 681/1352] 10.222.0.45 () {54 vars in 921 bytes} [Wed Jun 15 14:34:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:58.593453 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc70b5-20bb-4f72-aace-aed8a6d881f7 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:58.593918 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc70b5-20bb-4f72-aace-aed8a6d881f7 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:58.594006 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc70b5-20bb-4f72-aace-aed8a6d881f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:58.613479 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-75fc70b5-20bb-4f72-aace-aed8a6d881f7 None None] MFA Rules not processed for user `f4128111a9aa450abb2146a5d9926dec`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:58.614683 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-75fc70b5-20bb-4f72-aace-aed8a6d881f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:58.623981 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 672/1353] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Jun 15 14:34:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:34:58.789276 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=13 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:00.089978 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be538347-d33a-4b82-b17a-c312e4a91e4b None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:00.090589 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be538347-d33a-4b82-b17a-c312e4a91e4b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:00.090589 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be538347-d33a-4b82-b17a-c312e4a91e4b None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:00.092141 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 682/1354] 10.222.0.45 () {54 vars in 921 bytes} [Wed Jun 15 14:35:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:00.097143 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28e978-afd2-4e5c-adf9-8f911fdedf3f None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:00.097556 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28e978-afd2-4e5c-adf9-8f911fdedf3f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:00.097989 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28e978-afd2-4e5c-adf9-8f911fdedf3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:00.116358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-cd28e978-afd2-4e5c-adf9-8f911fdedf3f None None] MFA Rules not processed for user `f4128111a9aa450abb2146a5d9926dec`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:00.117533 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-cd28e978-afd2-4e5c-adf9-8f911fdedf3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:00.126340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 673/1355] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Jun 15 14:35:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:35:03.945657 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ef062-c23c-445c-97bc-dbd96b8fcdd2 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:03.946169 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ef062-c23c-445c-97bc-dbd96b8fcdd2 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:03.946169 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ef062-c23c-445c-97bc-dbd96b8fcdd2 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:03.947581 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 683/1356] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:35:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:03.952954 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d0f3e-521f-40e5-a773-1e3ee45a7362 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:03.953342 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d0f3e-521f-40e5-a773-1e3ee45a7362 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:03.953498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d0f3e-521f-40e5-a773-1e3ee45a7362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:03.973796 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-c20d0f3e-521f-40e5-a773-1e3ee45a7362 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:03.974889 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c20d0f3e-521f-40e5-a773-1e3ee45a7362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:03.983634 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 674/1357] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:35:04.038309 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302027a-4c85-4b9b-b4cb-6db6d437eeb6 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:04.039123 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302027a-4c85-4b9b-b4cb-6db6d437eeb6 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:04.039587 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302027a-4c85-4b9b-b4cb-6db6d437eeb6 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:04.042369 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 684/1358] 10.222.0.45 () {54 vars in 961 bytes} [Wed Jun 15 14:35:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:04.049373 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907b7cc-cafc-47b3-b91b-6d20604688cd None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:04.050029 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907b7cc-cafc-47b3-b91b-6d20604688cd None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:04.050616 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907b7cc-cafc-47b3-b91b-6d20604688cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:04.096980 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:04.106675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-3907b7cc-cafc-47b3-b91b-6d20604688cd None None] MFA Rules not processed for user `f3185bd2458c4e22830975700888e9be`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:04.107910 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-3907b7cc-cafc-47b3-b91b-6d20604688cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:04.197225 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 675/1359] 10.222.0.45 () {58 vars in 1058 bytes} [Wed Jun 15 14:35:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:04.204786 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9302027a-4c85-4b9b-b4cb-6db6d437eeb6 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:04.219725 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9302027a-4c85-4b9b-b4cb-6db6d437eeb6 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:04.226182 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db31f13-faf1-4c34-93fb-17c3d11698b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:04.227140 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db31f13-faf1-4c34-93fb-17c3d11698b1 service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:04.227642 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db31f13-faf1-4c34-93fb-17c3d11698b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:04.234626 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db31f13-faf1-4c34-93fb-17c3d11698b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:04.255472 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db31f13-faf1-4c34-93fb-17c3d11698b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:04.257094 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db31f13-faf1-4c34-93fb-17c3d11698b1 service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:04.267544 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 685/1360] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:35:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:35:06.593949 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c041d8-16b4-4564-ba1e-bf2b9a714a90 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:06.594488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c041d8-16b4-4564-ba1e-bf2b9a714a90 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:06.594488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c041d8-16b4-4564-ba1e-bf2b9a714a90 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:06.596063 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 676/1361] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:35:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:06.601221 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1bcd6-55e6-4953-aca7-f9a689131759 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:06.601694 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1bcd6-55e6-4953-aca7-f9a689131759 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:06.601787 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1bcd6-55e6-4953-aca7-f9a689131759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:06.620882 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-eab1bcd6-55e6-4953-aca7-f9a689131759 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:06.622126 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-eab1bcd6-55e6-4953-aca7-f9a689131759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:06.631977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 686/1362] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:06.679417 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c041d8-16b4-4564-ba1e-bf2b9a714a90 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:06.694631 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c041d8-16b4-4564-ba1e-bf2b9a714a90 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:06.701118 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b140b-0511-464d-80fa-3d0ad027d898 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:06.701992 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b140b-0511-464d-80fa-3d0ad027d898 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:06.702427 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b140b-0511-464d-80fa-3d0ad027d898 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:06.702832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b140b-0511-464d-80fa-3d0ad027d898 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:06.717197 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b140b-0511-464d-80fa-3d0ad027d898 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:06.718660 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b140b-0511-464d-80fa-3d0ad027d898 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:06.732766 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 677/1363] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:35:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:35:09.205064 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97239d82-6353-4f6e-a315-4d12b0db4a90 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:09.205603 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97239d82-6353-4f6e-a315-4d12b0db4a90 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:09.205603 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97239d82-6353-4f6e-a315-4d12b0db4a90 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:09.207006 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 687/1364] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:35:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:09.212198 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f07f9c-d8dd-4d7b-8f3f-5b84a42a3dd1 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:09.213226 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f07f9c-d8dd-4d7b-8f3f-5b84a42a3dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:09.213400 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f07f9c-d8dd-4d7b-8f3f-5b84a42a3dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:09.232033 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-f5f07f9c-d8dd-4d7b-8f3f-5b84a42a3dd1 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:09.233231 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f5f07f9c-d8dd-4d7b-8f3f-5b84a42a3dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:09.242328 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 678/1365] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:35:09.290112 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09d0e8-ea2c-4c5c-9dab-90fbecb889ed None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:09.290322 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09d0e8-ea2c-4c5c-9dab-90fbecb889ed None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:09.290599 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09d0e8-ea2c-4c5c-9dab-90fbecb889ed None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:09.292025 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 688/1366] 10.222.0.45 () {54 vars in 961 bytes} [Wed Jun 15 14:35:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:09.297744 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31882435-c924-4c30-bac2-66c25d8f3f3e None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:09.298028 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31882435-c924-4c30-bac2-66c25d8f3f3e None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:09.298303 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31882435-c924-4c30-bac2-66c25d8f3f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:09.316977 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-31882435-c924-4c30-bac2-66c25d8f3f3e None None] MFA Rules not processed for user `f3185bd2458c4e22830975700888e9be`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:09.318188 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-31882435-c924-4c30-bac2-66c25d8f3f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:09.327945 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 679/1367] 10.222.0.45 () {58 vars in 1058 bytes} [Wed Jun 15 14:35:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:09.335673 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09d0e8-ea2c-4c5c-9dab-90fbecb889ed None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:09.349924 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09d0e8-ea2c-4c5c-9dab-90fbecb889ed None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:09.355509 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f3e2f9-ad20-4dd0-94e1-06ba57371d74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:09.356148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f3e2f9-ad20-4dd0-94e1-06ba57371d74 service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:09.356432 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f3e2f9-ad20-4dd0-94e1-06ba57371d74 service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:09.362249 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f3e2f9-ad20-4dd0-94e1-06ba57371d74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:09.372247 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f3e2f9-ad20-4dd0-94e1-06ba57371d74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:09.374098 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f3e2f9-ad20-4dd0-94e1-06ba57371d74 service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:09.383687 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 689/1368] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:35:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:35:11.613472 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43719c-c864-4aea-9e4e-a7c4c95bbfbe None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:11.614102 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43719c-c864-4aea-9e4e-a7c4c95bbfbe None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:11.614102 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43719c-c864-4aea-9e4e-a7c4c95bbfbe None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:11.615485 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 680/1369] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:35:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:11.621268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11beff1c-de31-48ab-a970-b099c8a95782 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:11.621705 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11beff1c-de31-48ab-a970-b099c8a95782 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:11.621822 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11beff1c-de31-48ab-a970-b099c8a95782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:11.641519 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-11beff1c-de31-48ab-a970-b099c8a95782 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:11.643073 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-11beff1c-de31-48ab-a970-b099c8a95782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:11.651588 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 690/1370] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:11.696366 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43719c-c864-4aea-9e4e-a7c4c95bbfbe None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:11.714330 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43719c-c864-4aea-9e4e-a7c4c95bbfbe None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:11.726000 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162cd2b-0c58-4b99-9338-e45dca25ca65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:11.727260 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162cd2b-0c58-4b99-9338-e45dca25ca65 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:11.728514 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162cd2b-0c58-4b99-9338-e45dca25ca65 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:11.729129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162cd2b-0c58-4b99-9338-e45dca25ca65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:11.739231 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0162cd2b-0c58-4b99-9338-e45dca25ca65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:11.741213 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0162cd2b-0c58-4b99-9338-e45dca25ca65 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:11.753401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 681/1371] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:35:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:35:12.140926 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760a026-7441-44df-becd-2103b6b6f08e None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:12.141210 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760a026-7441-44df-becd-2103b6b6f08e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:12.141757 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760a026-7441-44df-becd-2103b6b6f08e None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:12.143279 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 691/1372] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:35:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:12.148768 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e28c7-2246-4f79-8376-204b2b031743 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:12.149066 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e28c7-2246-4f79-8376-204b2b031743 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:12.149349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e28c7-2246-4f79-8376-204b2b031743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:12.168615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-9c3e28c7-2246-4f79-8376-204b2b031743 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:12.169808 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-9c3e28c7-2246-4f79-8376-204b2b031743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:12.183039 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 682/1373] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:12.485626 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd4bf8-74b2-4285-8fc1-7fe254d99ec4 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:12.487097 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd4bf8-74b2-4285-8fc1-7fe254d99ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:12.487097 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd4bf8-74b2-4285-8fc1-7fe254d99ec4 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:12.488229 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 692/1374] 10.222.0.45 () {54 vars in 963 bytes} [Wed Jun 15 14:35:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:12.494265 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a68c29-7058-4268-8511-0960cf8a0bb5 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:12.494900 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a68c29-7058-4268-8511-0960cf8a0bb5 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:12.495655 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a68c29-7058-4268-8511-0960cf8a0bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:12.549198 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-e6a68c29-7058-4268-8511-0960cf8a0bb5 None None] MFA Rules not processed for user `b47cec76cb1c4afb8e011ff04a885f96`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:12.550404 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e6a68c29-7058-4268-8511-0960cf8a0bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:12.613799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 683/1375] 10.222.0.45 () {58 vars in 1060 bytes} [Wed Jun 15 14:35:12 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:12.621211 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd4bf8-74b2-4285-8fc1-7fe254d99ec4 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:12.631935 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd4bf8-74b2-4285-8fc1-7fe254d99ec4 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:12.636950 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c774413b-c934-4580-9dc9-27e764777df4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:12.637842 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774413b-c934-4580-9dc9-27e764777df4 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:12.638437 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774413b-c934-4580-9dc9-27e764777df4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:12.638901 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774413b-c934-4580-9dc9-27e764777df4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:12.649577 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c774413b-c934-4580-9dc9-27e764777df4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:12.652300 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c774413b-c934-4580-9dc9-27e764777df4 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:12.663885 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 693/1376] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:35:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:35:14.056606 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6386f63-fc1e-48e9-98f1-b6fb354099ca None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:14.058026 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6386f63-fc1e-48e9-98f1-b6fb354099ca None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:14.058026 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6386f63-fc1e-48e9-98f1-b6fb354099ca None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:14.058593 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 684/1377] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:35:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:14.063929 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0054532-7a4e-4006-9eaa-d3a833bf3ed7 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:14.064336 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0054532-7a4e-4006-9eaa-d3a833bf3ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:14.064485 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0054532-7a4e-4006-9eaa-d3a833bf3ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:14.082377 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-e0054532-7a4e-4006-9eaa-d3a833bf3ed7 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:14.083585 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e0054532-7a4e-4006-9eaa-d3a833bf3ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:14.094802 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 694/1378] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:14.140178 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6386f63-fc1e-48e9-98f1-b6fb354099ca None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:14.155321 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6386f63-fc1e-48e9-98f1-b6fb354099ca None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:14.160874 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc3d05-7b70-4593-bfa7-0899edafb0a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:14.161513 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc3d05-7b70-4593-bfa7-0899edafb0a9 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:14.161800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc3d05-7b70-4593-bfa7-0899edafb0a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:14.162081 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc3d05-7b70-4593-bfa7-0899edafb0a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:14.170916 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc3d05-7b70-4593-bfa7-0899edafb0a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:14.172521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc3d05-7b70-4593-bfa7-0899edafb0a9 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:14.176864 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:14.184620 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 685/1379] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:35:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:35:16.416821 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd9f61b-c4ba-410f-9e84-caaf8cfb1b4c None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:16.418214 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd9f61b-c4ba-410f-9e84-caaf8cfb1b4c None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:16.418719 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd9f61b-c4ba-410f-9e84-caaf8cfb1b4c None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:16.423118 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 695/1380] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:35:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:16.427257 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e8dd4-fe2f-49c8-b67f-ff0bee14eeb5 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:16.428068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e8dd4-fe2f-49c8-b67f-ff0bee14eeb5 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:16.428509 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e8dd4-fe2f-49c8-b67f-ff0bee14eeb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:16.454855 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-7e5e8dd4-fe2f-49c8-b67f-ff0bee14eeb5 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:16.456520 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-7e5e8dd4-fe2f-49c8-b67f-ff0bee14eeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:16.465795 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 686/1381] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:16.518429 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd9f61b-c4ba-410f-9e84-caaf8cfb1b4c None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:16.533404 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd9f61b-c4ba-410f-9e84-caaf8cfb1b4c None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:16.538944 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735b155-9626-4542-b83f-b0082abe10b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:16.539631 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735b155-9626-4542-b83f-b0082abe10b6 service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:16.539932 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735b155-9626-4542-b83f-b0082abe10b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:16.540744 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735b155-9626-4542-b83f-b0082abe10b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:16.550387 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6735b155-9626-4542-b83f-b0082abe10b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:16.552019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6735b155-9626-4542-b83f-b0082abe10b6 service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:16.563426 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 696/1382] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:35:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:35:18.832062 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292281e-a966-483c-af81-326bac50b1e1 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:18.832961 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292281e-a966-483c-af81-326bac50b1e1 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:18.833408 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292281e-a966-483c-af81-326bac50b1e1 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:18.835320 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 687/1383] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:35:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:18.841659 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b53de21-2337-4f04-8c96-9749ec009c8b None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:18.842085 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b53de21-2337-4f04-8c96-9749ec009c8b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:18.842273 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b53de21-2337-4f04-8c96-9749ec009c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:18.862650 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-8b53de21-2337-4f04-8c96-9749ec009c8b None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:18.863883 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-8b53de21-2337-4f04-8c96-9749ec009c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:18.884138 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 697/1384] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:18.918158 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292281e-a966-483c-af81-326bac50b1e1 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:18.934554 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292281e-a966-483c-af81-326bac50b1e1 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:18.941013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb7677-bbde-4811-a1b6-7ecf414edf75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:18.941663 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb7677-bbde-4811-a1b6-7ecf414edf75 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:18.941953 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb7677-bbde-4811-a1b6-7ecf414edf75 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:18.942234 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb7677-bbde-4811-a1b6-7ecf414edf75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:18.952706 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb7677-bbde-4811-a1b6-7ecf414edf75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:18.954296 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb7677-bbde-4811-a1b6-7ecf414edf75 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:18.968249 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 688/1385] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:35:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:35:21.539481 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d51627-a5c3-4645-92c6-8979d573b8ce None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:21.540565 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d51627-a5c3-4645-92c6-8979d573b8ce None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:21.540565 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d51627-a5c3-4645-92c6-8979d573b8ce None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:21.541934 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 698/1386] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:35:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:21.548017 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ff451-08d1-4177-9870-f9a62eb44e3d None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:21.548698 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ff451-08d1-4177-9870-f9a62eb44e3d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:21.548698 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ff451-08d1-4177-9870-f9a62eb44e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:21.569199 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-2c6ff451-08d1-4177-9870-f9a62eb44e3d None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:21.570400 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-2c6ff451-08d1-4177-9870-f9a62eb44e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:21.578796 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 689/1387] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:21.638013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d51627-a5c3-4645-92c6-8979d573b8ce None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:21.653210 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d51627-a5c3-4645-92c6-8979d573b8ce None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:21.658802 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da68b7-4659-41dd-b5ca-48834014dc44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:21.659471 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da68b7-4659-41dd-b5ca-48834014dc44 service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:21.659758 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da68b7-4659-41dd-b5ca-48834014dc44 service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:21.660041 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da68b7-4659-41dd-b5ca-48834014dc44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:21.668385 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da68b7-4659-41dd-b5ca-48834014dc44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:21.670026 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da68b7-4659-41dd-b5ca-48834014dc44 service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:21.682470 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 699/1388] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:35:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:35:23.960688 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3030cea-6fb5-4c93-b186-4c50340172c8 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:23.961314 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3030cea-6fb5-4c93-b186-4c50340172c8 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:23.961314 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3030cea-6fb5-4c93-b186-4c50340172c8 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:23.963798 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 690/1389] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:35:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:23.969775 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80bd527-6841-4f8e-bd5a-74ba7ce79efa None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:23.970428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80bd527-6841-4f8e-bd5a-74ba7ce79efa None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:23.970428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80bd527-6841-4f8e-bd5a-74ba7ce79efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:23.989404 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-a80bd527-6841-4f8e-bd5a-74ba7ce79efa None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:23.991027 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a80bd527-6841-4f8e-bd5a-74ba7ce79efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:24.001863 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 700/1390] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:24.053794 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3030cea-6fb5-4c93-b186-4c50340172c8 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:24.070898 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3030cea-6fb5-4c93-b186-4c50340172c8 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:24.079793 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a00053-2693-45b5-8973-4ddb42db04f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:24.080841 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a00053-2693-45b5-8973-4ddb42db04f0 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:24.081399 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a00053-2693-45b5-8973-4ddb42db04f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:24.081886 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a00053-2693-45b5-8973-4ddb42db04f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:24.092012 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a00053-2693-45b5-8973-4ddb42db04f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:24.093627 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a00053-2693-45b5-8973-4ddb42db04f0 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:24.104685 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 691/1391] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:35:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:35:26.792077 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29b52e-8d21-4632-8eb5-4cb5789540b0 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:26.792605 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29b52e-8d21-4632-8eb5-4cb5789540b0 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:26.792788 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29b52e-8d21-4632-8eb5-4cb5789540b0 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:26.794541 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 701/1392] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:35:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:26.801346 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624bbaf-2096-40a7-8afc-bb4fa8d6103b None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:26.802016 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624bbaf-2096-40a7-8afc-bb4fa8d6103b None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:26.802016 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624bbaf-2096-40a7-8afc-bb4fa8d6103b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:26.821373 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-c624bbaf-2096-40a7-8afc-bb4fa8d6103b None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:26.822603 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c624bbaf-2096-40a7-8afc-bb4fa8d6103b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:26.830964 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 692/1393] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:35:26.890165 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29b52e-8d21-4632-8eb5-4cb5789540b0 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:26.907947 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29b52e-8d21-4632-8eb5-4cb5789540b0 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:26.913744 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564fa333-8412-4fe8-9cda-a5aafc0ba634 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:26.914401 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564fa333-8412-4fe8-9cda-a5aafc0ba634 service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:26.914684 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564fa333-8412-4fe8-9cda-a5aafc0ba634 service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:26.915010 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564fa333-8412-4fe8-9cda-a5aafc0ba634 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:26.924493 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564fa333-8412-4fe8-9cda-a5aafc0ba634 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:26.926113 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564fa333-8412-4fe8-9cda-a5aafc0ba634 service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:26.937014 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 702/1394] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:35:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:35:29.439034 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce86eed-6950-40d3-baa8-da0a77cd0626 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:29.439679 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce86eed-6950-40d3-baa8-da0a77cd0626 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:29.439679 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce86eed-6950-40d3-baa8-da0a77cd0626 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:29.440971 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 693/1395] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:29.447968 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577421c-3cfd-41c6-bfee-d1bf79fe2a47 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:29.448446 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577421c-3cfd-41c6-bfee-d1bf79fe2a47 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:29.448729 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577421c-3cfd-41c6-bfee-d1bf79fe2a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:29.455706 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=52 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:29.469243 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-8577421c-3cfd-41c6-bfee-d1bf79fe2a47 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:29.470518 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-8577421c-3cfd-41c6-bfee-d1bf79fe2a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:29.479722 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 703/1396] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:29.526356 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce86eed-6950-40d3-baa8-da0a77cd0626 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:29.544244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce86eed-6950-40d3-baa8-da0a77cd0626 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:29.549921 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc545e9-bb98-4da1-9d12-660c5ff6ad58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:29.550691 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc545e9-bb98-4da1-9d12-660c5ff6ad58 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:29.550987 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc545e9-bb98-4da1-9d12-660c5ff6ad58 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:29.551374 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc545e9-bb98-4da1-9d12-660c5ff6ad58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:29.560805 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc545e9-bb98-4da1-9d12-660c5ff6ad58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:29.562390 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc545e9-bb98-4da1-9d12-660c5ff6ad58 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:29.573122 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 694/1397] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:35:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:35:31.587032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c97697-a134-4ad9-a0f6-96fc5edc955a None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:31.587557 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c97697-a134-4ad9-a0f6-96fc5edc955a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:31.587639 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c97697-a134-4ad9-a0f6-96fc5edc955a None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:31.589281 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 704/1398] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:35:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:31.594497 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca52372-1277-44b5-b874-76ba638037f7 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:31.594906 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca52372-1277-44b5-b874-76ba638037f7 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:31.595111 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca52372-1277-44b5-b874-76ba638037f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:31.619553 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-9ca52372-1277-44b5-b874-76ba638037f7 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:31.620785 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-9ca52372-1277-44b5-b874-76ba638037f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:31.629145 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 695/1399] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:31.709871 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2efa65-b789-45be-bdc7-e05a78d5fd2a None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:31.710158 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2efa65-b789-45be-bdc7-e05a78d5fd2a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:31.710437 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2efa65-b789-45be-bdc7-e05a78d5fd2a None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:31.712355 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 705/1400] 10.222.0.45 () {54 vars in 962 bytes} [Wed Jun 15 14:35:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:31.717391 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70b80f-b450-4a91-a2ef-86ce992d171f None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:31.717711 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70b80f-b450-4a91-a2ef-86ce992d171f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:31.717941 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70b80f-b450-4a91-a2ef-86ce992d171f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:31.778974 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-6c70b80f-b450-4a91-a2ef-86ce992d171f None None] MFA Rules not processed for user `e4f10214b2bb4eafb3dff7adf71b9bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:31.780279 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-6c70b80f-b450-4a91-a2ef-86ce992d171f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:31.803989 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141b65e-e2b2-4f70-a6e3-b3ff888aa7d3 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:31.804282 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141b65e-e2b2-4f70-a6e3-b3ff888aa7d3 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:31.804562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141b65e-e2b2-4f70-a6e3-b3ff888aa7d3 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:31.806491 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 706/1401] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:35:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:31.812229 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5c40d-b191-45ca-967f-93529d47f7b2 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:31.812515 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5c40d-b191-45ca-967f-93529d47f7b2 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:31.812794 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5c40d-b191-45ca-967f-93529d47f7b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:31.830713 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-1fd5c40d-b191-45ca-967f-93529d47f7b2 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:31.831958 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1fd5c40d-b191-45ca-967f-93529d47f7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:31.840622 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 707/1402] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:35:31.848443 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 696/1403] 10.222.0.45 () {58 vars in 1059 bytes} [Wed Jun 15 14:35:31 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:31.856713 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5c40d-b191-45ca-967f-93529d47f7b2 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:31.866655 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5c40d-b191-45ca-967f-93529d47f7b2 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:31.870304 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ca639-664f-47a2-8036-c16bde7a36be service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f10214b2bb4eafb3dff7adf71b9bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:31.871186 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ca639-664f-47a2-8036-c16bde7a36be service glance] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:31.871306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ca639-664f-47a2-8036-c16bde7a36be service glance] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:31.871636 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ca639-664f-47a2-8036-c16bde7a36be service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:31.882245 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ca639-664f-47a2-8036-c16bde7a36be service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:31.883761 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ca639-664f-47a2-8036-c16bde7a36be service glance] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:31.885601 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70b80f-b450-4a91-a2ef-86ce992d171f None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:31.894335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 708/1404] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:35:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:35:31.899238 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70b80f-b450-4a91-a2ef-86ce992d171f None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:31.904963 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5502e588-bad8-4004-a68d-1b85f902220c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:31.905613 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502e588-bad8-4004-a68d-1b85f902220c service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:31.905900 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502e588-bad8-4004-a68d-1b85f902220c service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:31.906181 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502e588-bad8-4004-a68d-1b85f902220c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:31.916482 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5502e588-bad8-4004-a68d-1b85f902220c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:31.918671 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5502e588-bad8-4004-a68d-1b85f902220c service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:31.932848 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 697/1405] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:35:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:35:32.651781 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16480e48-55ce-40af-8ae5-da822e002143 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:32.652262 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16480e48-55ce-40af-8ae5-da822e002143 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:32.652376 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16480e48-55ce-40af-8ae5-da822e002143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:32.703356 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-16480e48-55ce-40af-8ae5-da822e002143 None None] MFA Rules not processed for user `bf393488726e49338e70017f5cb7c4be`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:32.704597 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-16480e48-55ce-40af-8ae5-da822e002143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:32.773004 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 709/1406] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Jun 15 14:35:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:32.820317 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfacb8fd-994a-40f9-a093-f8d30ce43e73 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:32.820815 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfacb8fd-994a-40f9-a093-f8d30ce43e73 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:32.820909 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfacb8fd-994a-40f9-a093-f8d30ce43e73 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:32.822280 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 698/1407] 10.222.0.45 () {54 vars in 942 bytes} [Wed Jun 15 14:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:32.828635 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a3e38-14e8-456e-90b0-f0c62350012f None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:32.828930 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a3e38-14e8-456e-90b0-f0c62350012f None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:32.829205 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a3e38-14e8-456e-90b0-f0c62350012f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:32.884788 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-df2a3e38-14e8-456e-90b0-f0c62350012f None None] MFA Rules not processed for user `641a81b3140f47119b5c2323a97a6681`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:32.886080 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-df2a3e38-14e8-456e-90b0-f0c62350012f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:32.994502 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 710/1408] 10.222.0.45 () {58 vars in 1039 bytes} [Wed Jun 15 14:35:32 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:33.004108 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfacb8fd-994a-40f9-a093-f8d30ce43e73 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:33.018641 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfacb8fd-994a-40f9-a093-f8d30ce43e73 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:33.024489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5992747-2a4c-46a4-a942-b9bb7c87635d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '641a81b3140f47119b5c2323a97a6681', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:33.025347 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5992747-2a4c-46a4-a942-b9bb7c87635d service swift] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:33.025630 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5992747-2a4c-46a4-a942-b9bb7c87635d service swift] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:33.025895 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5992747-2a4c-46a4-a942-b9bb7c87635d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:33.035982 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5992747-2a4c-46a4-a942-b9bb7c87635d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:33.038963 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5992747-2a4c-46a4-a942-b9bb7c87635d service swift] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:33.048484 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 699/1409] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Jun 15 14:35:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:35:42.991601 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=95 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:43.044844 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=81 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:56.312790 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee66fc6-19ac-4bac-b27f-259e2b4730f9 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:56.313370 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee66fc6-19ac-4bac-b27f-259e2b4730f9 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:56.313488 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee66fc6-19ac-4bac-b27f-259e2b4730f9 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:56.315169 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 711/1410] 10.222.0.45 () {54 vars in 918 bytes} [Wed Jun 15 14:35:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:35:56.322073 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4ad65-fdda-4ede-8d14-bb066d5533aa None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:56.322902 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4ad65-fdda-4ede-8d14-bb066d5533aa None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:56.322902 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4ad65-fdda-4ede-8d14-bb066d5533aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:56.344403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-49e4ad65-fdda-4ede-8d14-bb066d5533aa None None] MFA Rules not processed for user `f4128111a9aa450abb2146a5d9926dec`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:56.345870 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-49e4ad65-fdda-4ede-8d14-bb066d5533aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:56.356620 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 700/1411] 10.222.0.45 () {58 vars in 1015 bytes} [Wed Jun 15 14:35:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:35:56.373560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56843f-3b29-4c4c-855c-231fb6f66875 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:56.373864 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56843f-3b29-4c4c-855c-231fb6f66875 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:56.374237 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56843f-3b29-4c4c-855c-231fb6f66875 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:56.376049 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 712/1412] 10.222.0.45 () {54 vars in 918 bytes} [Wed Jun 15 14:35:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:56.384916 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9297520-ee87-44b8-a341-8cbc835835ef None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:56.385458 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9297520-ee87-44b8-a341-8cbc835835ef None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:56.385961 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9297520-ee87-44b8-a341-8cbc835835ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:56.418577 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-c9297520-ee87-44b8-a341-8cbc835835ef None None] MFA Rules not processed for user `f3185bd2458c4e22830975700888e9be`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:56.420918 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c9297520-ee87-44b8-a341-8cbc835835ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:56.436618 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 701/1413] 10.222.0.45 () {58 vars in 1015 bytes} [Wed Jun 15 14:35:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:59.315453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607613fb-5400-4e22-8bb3-ff4596fe1a28 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:59.316047 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607613fb-5400-4e22-8bb3-ff4596fe1a28 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:59.316345 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607613fb-5400-4e22-8bb3-ff4596fe1a28 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:59.318687 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 713/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:35:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:59.326379 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cb86b-f507-4667-892a-aca90069de22 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:59.327155 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cb86b-f507-4667-892a-aca90069de22 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:59.327155 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cb86b-f507-4667-892a-aca90069de22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:59.346631 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-636cb86b-f507-4667-892a-aca90069de22 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:59.347942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-636cb86b-f507-4667-892a-aca90069de22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:59.357177 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 702/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:35:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:59.430350 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607613fb-5400-4e22-8bb3-ff4596fe1a28 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:59.452428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607613fb-5400-4e22-8bb3-ff4596fe1a28 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:59.460603 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6ef5f-7da3-4c92-b228-28c71d302bed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:59.461405 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6ef5f-7da3-4c92-b228-28c71d302bed service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:59.461802 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6ef5f-7da3-4c92-b228-28c71d302bed service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:59.462166 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6ef5f-7da3-4c92-b228-28c71d302bed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:59.471900 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef6ef5f-7da3-4c92-b228-28c71d302bed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:59.473682 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef6ef5f-7da3-4c92-b228-28c71d302bed service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:59.484469 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 714/1416] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:35:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:36:00.487600 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac5280-1776-43ec-b91a-d50d6aa755c6 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:00.488174 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac5280-1776-43ec-b91a-d50d6aa755c6 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:00.488174 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac5280-1776-43ec-b91a-d50d6aa755c6 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:00.489712 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 703/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:36:00.496788 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17db18aa-d1fd-49b0-b861-9ed190ac9205 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:00.497332 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17db18aa-d1fd-49b0-b861-9ed190ac9205 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:00.497710 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17db18aa-d1fd-49b0-b861-9ed190ac9205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:00.522294 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-17db18aa-d1fd-49b0-b861-9ed190ac9205 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:00.523549 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-17db18aa-d1fd-49b0-b861-9ed190ac9205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:00.532242 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 715/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:36:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:36:00.686660 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac5280-1776-43ec-b91a-d50d6aa755c6 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:00.702821 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac5280-1776-43ec-b91a-d50d6aa755c6 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:00.709216 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063a7f0-b74f-4cca-967d-09c56dd82fe5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:00.710087 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063a7f0-b74f-4cca-967d-09c56dd82fe5 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:00.710463 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063a7f0-b74f-4cca-967d-09c56dd82fe5 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:00.710932 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063a7f0-b74f-4cca-967d-09c56dd82fe5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:00.720927 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6063a7f0-b74f-4cca-967d-09c56dd82fe5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:00.722730 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6063a7f0-b74f-4cca-967d-09c56dd82fe5 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:00.735487 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 704/1419] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:36:01.813453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fabee3-1966-4bc7-b935-4e3a6082dd62 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:01.814308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fabee3-1966-4bc7-b935-4e3a6082dd62 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:01.814308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fabee3-1966-4bc7-b935-4e3a6082dd62 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:01.815614 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 716/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:36:01.824622 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dca7f-b176-4342-a8d9-ce12c71edb49 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:01.825439 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dca7f-b176-4342-a8d9-ce12c71edb49 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:01.825613 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dca7f-b176-4342-a8d9-ce12c71edb49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:01.853773 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-f96dca7f-b176-4342-a8d9-ce12c71edb49 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:01.855195 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f96dca7f-b176-4342-a8d9-ce12c71edb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:01.863552 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 705/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:36:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:36:02.039952 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fabee3-1966-4bc7-b935-4e3a6082dd62 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:02.056700 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fabee3-1966-4bc7-b935-4e3a6082dd62 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:02.062344 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60581e4e-c204-457f-a289-05cd91450853 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:02.063031 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60581e4e-c204-457f-a289-05cd91450853 service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:02.063309 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60581e4e-c204-457f-a289-05cd91450853 service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:02.063588 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60581e4e-c204-457f-a289-05cd91450853 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:02.074157 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60581e4e-c204-457f-a289-05cd91450853 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:02.075846 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60581e4e-c204-457f-a289-05cd91450853 service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:02.087211 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 717/1422] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:36:03.509353 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e112c6e-a405-411a-b7ea-fbdce8329955 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:03.510163 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e112c6e-a405-411a-b7ea-fbdce8329955 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:03.510163 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e112c6e-a405-411a-b7ea-fbdce8329955 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:03.511866 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 706/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:36:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:03.517969 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ee94d-aee5-4c04-801b-f35e12e1686b None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:03.518786 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ee94d-aee5-4c04-801b-f35e12e1686b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:03.518786 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ee94d-aee5-4c04-801b-f35e12e1686b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:03.541109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-ab1ee94d-aee5-4c04-801b-f35e12e1686b None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:03.542451 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ab1ee94d-aee5-4c04-801b-f35e12e1686b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:03.551795 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 718/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:36:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:36:03.708992 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e112c6e-a405-411a-b7ea-fbdce8329955 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:03.724938 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e112c6e-a405-411a-b7ea-fbdce8329955 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:03.731795 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ebac6-951e-4106-ace7-3900e10eb310 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:03.732476 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ebac6-951e-4106-ace7-3900e10eb310 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:03.732798 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ebac6-951e-4106-ace7-3900e10eb310 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:03.733111 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ebac6-951e-4106-ace7-3900e10eb310 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:03.743051 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48ebac6-951e-4106-ace7-3900e10eb310 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:03.744777 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48ebac6-951e-4106-ace7-3900e10eb310 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:03.755573 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 707/1425] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:36:04.776770 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de701d-f8df-4228-b672-8eca93bac04f None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:04.777438 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de701d-f8df-4228-b672-8eca93bac04f None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:04.777438 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de701d-f8df-4228-b672-8eca93bac04f None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:04.779051 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 719/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:36:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:36:04.785732 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78fb8f-bcc0-45d0-9c4e-53d1227cb869 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:04.786354 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78fb8f-bcc0-45d0-9c4e-53d1227cb869 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:04.786819 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78fb8f-bcc0-45d0-9c4e-53d1227cb869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:04.812153 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-ba78fb8f-bcc0-45d0-9c4e-53d1227cb869 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:04.813438 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ba78fb8f-bcc0-45d0-9c4e-53d1227cb869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:04.821848 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 708/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:36:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:36:04.983750 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de701d-f8df-4228-b672-8eca93bac04f None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:04.999371 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de701d-f8df-4228-b672-8eca93bac04f None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:05.008634 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11ca4b0-1a03-4a18-8ba1-b43683f44922 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:05.009820 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ca4b0-1a03-4a18-8ba1-b43683f44922 service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:05.010283 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ca4b0-1a03-4a18-8ba1-b43683f44922 service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:05.010743 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ca4b0-1a03-4a18-8ba1-b43683f44922 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:05.025592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11ca4b0-1a03-4a18-8ba1-b43683f44922 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:05.028441 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11ca4b0-1a03-4a18-8ba1-b43683f44922 service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:05.043851 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 720/1428] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:36:06.012558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf5501-5527-44f9-b91f-bafd663da3dc None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:06.013352 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf5501-5527-44f9-b91f-bafd663da3dc None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:06.013352 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf5501-5527-44f9-b91f-bafd663da3dc None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:06.014935 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 709/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:36:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:36:06.020839 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4de7b-2c66-4543-b92f-1313bfc2a814 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:06.022177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4de7b-2c66-4543-b92f-1313bfc2a814 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:06.022177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4de7b-2c66-4543-b92f-1313bfc2a814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:06.044882 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-44f4de7b-2c66-4543-b92f-1313bfc2a814 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:06.046769 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-44f4de7b-2c66-4543-b92f-1313bfc2a814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:06.060160 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 721/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:36:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:36:06.291354 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf5501-5527-44f9-b91f-bafd663da3dc None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:06.307732 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf5501-5527-44f9-b91f-bafd663da3dc None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:06.313723 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c3594-7755-4b17-b18e-4da30fc02031 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:06.314426 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c3594-7755-4b17-b18e-4da30fc02031 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:06.314835 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c3594-7755-4b17-b18e-4da30fc02031 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:06.315220 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c3594-7755-4b17-b18e-4da30fc02031 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:06.325307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9c3594-7755-4b17-b18e-4da30fc02031 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:06.328385 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9c3594-7755-4b17-b18e-4da30fc02031 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:06.345832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 710/1431] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:36:07.252729 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0520a-87e7-413a-805f-0e02b2f45423 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:07.253275 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0520a-87e7-413a-805f-0e02b2f45423 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:07.253275 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0520a-87e7-413a-805f-0e02b2f45423 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:07.254656 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 722/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:36:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:36:07.261489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5fae20-bb98-43cd-bef8-ff574adbe19a None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:07.262306 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5fae20-bb98-43cd-bef8-ff574adbe19a None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:07.262464 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5fae20-bb98-43cd-bef8-ff574adbe19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:07.289603 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-bc5fae20-bb98-43cd-bef8-ff574adbe19a None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:07.291681 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-bc5fae20-bb98-43cd-bef8-ff574adbe19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:07.304731 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 711/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:36:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:36:07.489346 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc0520a-87e7-413a-805f-0e02b2f45423 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:07.506694 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc0520a-87e7-413a-805f-0e02b2f45423 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:07.512702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784bd8a-05c0-41dc-a9bd-d6135067df3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:07.513411 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784bd8a-05c0-41dc-a9bd-d6135067df3b service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:07.513756 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784bd8a-05c0-41dc-a9bd-d6135067df3b service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:07.514082 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784bd8a-05c0-41dc-a9bd-d6135067df3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:07.524212 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3784bd8a-05c0-41dc-a9bd-d6135067df3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:07.526040 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3784bd8a-05c0-41dc-a9bd-d6135067df3b service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:07.535919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 723/1434] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:36:08.445948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787249bf-a59d-4fa7-876c-7bc858caf105 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:08.446717 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787249bf-a59d-4fa7-876c-7bc858caf105 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:08.446908 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787249bf-a59d-4fa7-876c-7bc858caf105 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:08.449461 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 712/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:36:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:08.456566 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b312c-2179-4fc1-9be1-659d4ec5c4de None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:08.457264 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b312c-2179-4fc1-9be1-659d4ec5c4de None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:08.457264 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b312c-2179-4fc1-9be1-659d4ec5c4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:08.476547 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-835b312c-2179-4fc1-9be1-659d4ec5c4de None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:08.478067 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-835b312c-2179-4fc1-9be1-659d4ec5c4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:08.489493 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 724/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:36:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:36:08.651600 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787249bf-a59d-4fa7-876c-7bc858caf105 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:08.676254 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787249bf-a59d-4fa7-876c-7bc858caf105 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:08.686384 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08010c0f-421d-49a8-aa35-653907ac295d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:08.687506 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08010c0f-421d-49a8-aa35-653907ac295d service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:08.687992 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08010c0f-421d-49a8-aa35-653907ac295d service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:08.688473 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08010c0f-421d-49a8-aa35-653907ac295d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:08.704313 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08010c0f-421d-49a8-aa35-653907ac295d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:08.707194 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08010c0f-421d-49a8-aa35-653907ac295d service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:08.724620 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 713/1437] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:36:09.819389 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097f582-db88-47af-8809-402032e52bef None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:09.820261 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097f582-db88-47af-8809-402032e52bef None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:09.820261 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097f582-db88-47af-8809-402032e52bef None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:09.821773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 725/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:36:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:09.829939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae6551-16a7-43cb-8da3-912db4feb939 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:09.830779 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae6551-16a7-43cb-8da3-912db4feb939 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:09.830955 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae6551-16a7-43cb-8da3-912db4feb939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:09.862052 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-68ae6551-16a7-43cb-8da3-912db4feb939 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:09.864128 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-68ae6551-16a7-43cb-8da3-912db4feb939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:09.880032 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 714/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:36:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:36:10.137977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8097f582-db88-47af-8809-402032e52bef None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:10.154152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8097f582-db88-47af-8809-402032e52bef None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:10.160274 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef97eca-224e-4512-9569-96511189286f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:10.161032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef97eca-224e-4512-9569-96511189286f service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:10.161366 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef97eca-224e-4512-9569-96511189286f service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:10.161715 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef97eca-224e-4512-9569-96511189286f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:10.171853 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef97eca-224e-4512-9569-96511189286f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:10.173719 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef97eca-224e-4512-9569-96511189286f service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:10.185754 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 726/1440] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:36:12.633082 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ddc9a-33c2-4bea-8d82-8b33fe0376fb None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:12.633725 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ddc9a-33c2-4bea-8d82-8b33fe0376fb None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:12.633725 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ddc9a-33c2-4bea-8d82-8b33fe0376fb None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:12.635084 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 715/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:12.641584 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86010480-dfc6-475f-afdd-b01edaed27cd None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:12.642114 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86010480-dfc6-475f-afdd-b01edaed27cd None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:12.642212 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86010480-dfc6-475f-afdd-b01edaed27cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:12.666076 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-86010480-dfc6-475f-afdd-b01edaed27cd None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:12.667339 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-86010480-dfc6-475f-afdd-b01edaed27cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:12.676934 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 727/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:36:12.735490 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ddc9a-33c2-4bea-8d82-8b33fe0376fb None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:12.747936 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ddc9a-33c2-4bea-8d82-8b33fe0376fb None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:12.751620 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b5acb4-e3f6-4f9f-93c1-b2d86046ab27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:12.752262 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5acb4-e3f6-4f9f-93c1-b2d86046ab27 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:12.752543 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5acb4-e3f6-4f9f-93c1-b2d86046ab27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:12.752814 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5acb4-e3f6-4f9f-93c1-b2d86046ab27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:12.769005 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b5acb4-e3f6-4f9f-93c1-b2d86046ab27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:12.771346 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b5acb4-e3f6-4f9f-93c1-b2d86046ab27 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:12.787977 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 716/1443] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:36:15.336808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942f2c4-b6b8-4a20-9a79-5fea11bc9ea5 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:15.337597 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942f2c4-b6b8-4a20-9a79-5fea11bc9ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:15.337807 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942f2c4-b6b8-4a20-9a79-5fea11bc9ea5 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:15.342143 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 728/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:36:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:15.347234 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1dec63-5064-4a3f-8ea0-98fce632067a None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:15.347984 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1dec63-5064-4a3f-8ea0-98fce632067a None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:15.348145 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1dec63-5064-4a3f-8ea0-98fce632067a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:15.368338 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-9d1dec63-5064-4a3f-8ea0-98fce632067a None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:15.369611 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-9d1dec63-5064-4a3f-8ea0-98fce632067a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:15.378395 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 717/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:36:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:36:15.421294 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6942f2c4-b6b8-4a20-9a79-5fea11bc9ea5 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:15.423252 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6942f2c4-b6b8-4a20-9a79-5fea11bc9ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:15.437417 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6942f2c4-b6b8-4a20-9a79-5fea11bc9ea5 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:15.441297 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5972753-21f2-4725-bf10-8799c0b65f40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:15.446283 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5972753-21f2-4725-bf10-8799c0b65f40 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:15.446283 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5972753-21f2-4725-bf10-8799c0b65f40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:15.446283 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5972753-21f2-4725-bf10-8799c0b65f40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:15.453706 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5972753-21f2-4725-bf10-8799c0b65f40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:15.455204 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5972753-21f2-4725-bf10-8799c0b65f40 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:15.470110 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 729/1446] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:36:18.581779 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71dadf3-1ddd-48a0-becc-55ada2e7fb8c None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:18.582554 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71dadf3-1ddd-48a0-becc-55ada2e7fb8c None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:18.582554 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71dadf3-1ddd-48a0-becc-55ada2e7fb8c None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:18.584039 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 718/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:36:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:36:18.590191 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb82fa-bf5c-4208-bbae-7cdb56017071 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:18.590698 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb82fa-bf5c-4208-bbae-7cdb56017071 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:18.591189 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb82fa-bf5c-4208-bbae-7cdb56017071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:18.618368 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-24eb82fa-bf5c-4208-bbae-7cdb56017071 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:18.620223 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-24eb82fa-bf5c-4208-bbae-7cdb56017071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:18.629042 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 730/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:36:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:36:18.670343 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71dadf3-1ddd-48a0-becc-55ada2e7fb8c None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:18.682538 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71dadf3-1ddd-48a0-becc-55ada2e7fb8c None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:18.686403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3552f681-5407-4bab-9f8a-7c885d43122d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:18.687162 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552f681-5407-4bab-9f8a-7c885d43122d service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:18.687485 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552f681-5407-4bab-9f8a-7c885d43122d service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:18.687804 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552f681-5407-4bab-9f8a-7c885d43122d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:18.699523 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3552f681-5407-4bab-9f8a-7c885d43122d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:18.701415 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3552f681-5407-4bab-9f8a-7c885d43122d service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:18.716145 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 719/1449] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:36:21.174885 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce6c63-1791-47ef-89aa-5408c1811250 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:21.175717 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce6c63-1791-47ef-89aa-5408c1811250 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:21.175717 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce6c63-1791-47ef-89aa-5408c1811250 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:21.176863 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 731/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:36:21.183306 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0608fb6-2e7b-45f4-87c2-ee75985bfec5 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:21.184072 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0608fb6-2e7b-45f4-87c2-ee75985bfec5 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:21.184072 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0608fb6-2e7b-45f4-87c2-ee75985bfec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:21.204388 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-f0608fb6-2e7b-45f4-87c2-ee75985bfec5 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:21.205674 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f0608fb6-2e7b-45f4-87c2-ee75985bfec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:21.222568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 720/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:36:21.266095 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce6c63-1791-47ef-89aa-5408c1811250 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:21.280944 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce6c63-1791-47ef-89aa-5408c1811250 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:21.285196 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f7518f-bbd6-4ca6-88dc-5eb0ed7a282d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:21.285831 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f7518f-bbd6-4ca6-88dc-5eb0ed7a282d service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:21.286114 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f7518f-bbd6-4ca6-88dc-5eb0ed7a282d service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:21.286429 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f7518f-bbd6-4ca6-88dc-5eb0ed7a282d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:21.296280 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f7518f-bbd6-4ca6-88dc-5eb0ed7a282d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:21.297738 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f7518f-bbd6-4ca6-88dc-5eb0ed7a282d service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:21.309219 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 732/1452] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:36:23.721234 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a35c2f-af8a-4cc1-8349-2c15b5654ff7 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:23.722092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a35c2f-af8a-4cc1-8349-2c15b5654ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:23.722630 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a35c2f-af8a-4cc1-8349-2c15b5654ff7 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:23.725002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 721/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:36:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:23.734002 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c9668-0b59-4b20-909a-3d4941202b25 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:23.734625 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c9668-0b59-4b20-909a-3d4941202b25 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:23.735147 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c9668-0b59-4b20-909a-3d4941202b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:23.746063 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=61 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:23.767832 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-e27c9668-0b59-4b20-909a-3d4941202b25 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:23.769890 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e27c9668-0b59-4b20-909a-3d4941202b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:23.788937 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 733/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:36:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:36:23.828284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a35c2f-af8a-4cc1-8349-2c15b5654ff7 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:23.835196 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=64 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:23.841940 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a35c2f-af8a-4cc1-8349-2c15b5654ff7 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:23.845809 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d0f448-ef8b-46e5-aa78-040711d62b2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:23.846493 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0f448-ef8b-46e5-aa78-040711d62b2b service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:23.847400 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0f448-ef8b-46e5-aa78-040711d62b2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:23.847400 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0f448-ef8b-46e5-aa78-040711d62b2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:23.857733 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d0f448-ef8b-46e5-aa78-040711d62b2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:23.859225 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d0f448-ef8b-46e5-aa78-040711d62b2b service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:23.869676 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 722/1455] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:36:26.152720 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3294a94-ae1b-4bb5-89c2-8ff121ddedaf None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:26.153441 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3294a94-ae1b-4bb5-89c2-8ff121ddedaf None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:26.153441 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3294a94-ae1b-4bb5-89c2-8ff121ddedaf None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:26.154733 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 734/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:26.160590 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df711362-7b9f-4327-b2d0-a6377071d97f None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:26.161109 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df711362-7b9f-4327-b2d0-a6377071d97f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:26.161398 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df711362-7b9f-4327-b2d0-a6377071d97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:26.189337 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-df711362-7b9f-4327-b2d0-a6377071d97f None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:26.191583 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-df711362-7b9f-4327-b2d0-a6377071d97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:26.205721 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 723/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:36:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:36:26.261001 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3294a94-ae1b-4bb5-89c2-8ff121ddedaf None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:26.273216 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3294a94-ae1b-4bb5-89c2-8ff121ddedaf None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:26.277781 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43da88-47d6-4d6e-b7f2-5d8656ff54fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:26.278474 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43da88-47d6-4d6e-b7f2-5d8656ff54fb service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:26.278792 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43da88-47d6-4d6e-b7f2-5d8656ff54fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:26.279198 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43da88-47d6-4d6e-b7f2-5d8656ff54fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:26.289783 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc43da88-47d6-4d6e-b7f2-5d8656ff54fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:26.291259 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc43da88-47d6-4d6e-b7f2-5d8656ff54fb service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:26.301267 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 735/1458] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:36:28.778590 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50a844-ff25-4ec1-8a39-118c93c76a2e None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:28.779189 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50a844-ff25-4ec1-8a39-118c93c76a2e None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:28.779527 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50a844-ff25-4ec1-8a39-118c93c76a2e None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:28.781864 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 724/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:36:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:28.787763 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12120c-a773-4458-a140-3e312a71c7cf None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:28.788293 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12120c-a773-4458-a140-3e312a71c7cf None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:28.788438 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12120c-a773-4458-a140-3e312a71c7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:28.812863 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-be12120c-a773-4458-a140-3e312a71c7cf None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:28.814109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-be12120c-a773-4458-a140-3e312a71c7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:28.823114 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 736/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:36:28.861630 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc50a844-ff25-4ec1-8a39-118c93c76a2e None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:28.873700 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc50a844-ff25-4ec1-8a39-118c93c76a2e None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:28.877809 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d89898-32d6-454a-8b64-ca586e0df103 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:28.878543 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d89898-32d6-454a-8b64-ca586e0df103 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:28.878932 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d89898-32d6-454a-8b64-ca586e0df103 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:28.879396 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d89898-32d6-454a-8b64-ca586e0df103 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:28.889811 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d89898-32d6-454a-8b64-ca586e0df103 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:28.891786 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d89898-32d6-454a-8b64-ca586e0df103 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:28.906184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 725/1461] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:36:31.376232 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd6940-84c3-4c8c-b4fe-235be8e38a3e None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:31.376887 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd6940-84c3-4c8c-b4fe-235be8e38a3e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:31.376887 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd6940-84c3-4c8c-b4fe-235be8e38a3e None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:31.378665 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 737/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:36:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:31.385813 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d5ab1-ff14-40c2-895b-3a2941af336b None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:31.386323 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d5ab1-ff14-40c2-895b-3a2941af336b None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:31.386462 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d5ab1-ff14-40c2-895b-3a2941af336b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:31.407111 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-765d5ab1-ff14-40c2-895b-3a2941af336b None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:31.408399 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-765d5ab1-ff14-40c2-895b-3a2941af336b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:31.417170 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 726/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:36:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:36:31.457006 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd6940-84c3-4c8c-b4fe-235be8e38a3e None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:31.469086 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd6940-84c3-4c8c-b4fe-235be8e38a3e None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:31.472951 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab6ecb-dafa-4b2d-9aa9-35176031b305 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:31.473610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab6ecb-dafa-4b2d-9aa9-35176031b305 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:31.473897 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab6ecb-dafa-4b2d-9aa9-35176031b305 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:31.474176 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab6ecb-dafa-4b2d-9aa9-35176031b305 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:31.486102 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ab6ecb-dafa-4b2d-9aa9-35176031b305 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:31.487613 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ab6ecb-dafa-4b2d-9aa9-35176031b305 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:31.498007 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 738/1464] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:36:33.756579 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e64752-cc9a-45a6-9bcb-240e0c3712aa None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:33.757118 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e64752-cc9a-45a6-9bcb-240e0c3712aa None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:33.757118 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e64752-cc9a-45a6-9bcb-240e0c3712aa None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:33.758572 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 727/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:36:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:33.765056 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba55460-6c41-4976-ab4b-c0d9fbe12576 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:33.765923 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba55460-6c41-4976-ab4b-c0d9fbe12576 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:33.765923 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba55460-6c41-4976-ab4b-c0d9fbe12576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:33.786217 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-7ba55460-6c41-4976-ab4b-c0d9fbe12576 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:33.787511 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-7ba55460-6c41-4976-ab4b-c0d9fbe12576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:33.796940 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 739/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:36:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:36:33.852547 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e64752-cc9a-45a6-9bcb-240e0c3712aa None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:33.870856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e64752-cc9a-45a6-9bcb-240e0c3712aa None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:33.875374 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e01ed6-20c2-4432-a044-cf8e8a6bcb49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:33.876131 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e01ed6-20c2-4432-a044-cf8e8a6bcb49 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:33.876596 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e01ed6-20c2-4432-a044-cf8e8a6bcb49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:33.876881 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e01ed6-20c2-4432-a044-cf8e8a6bcb49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:33.893130 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e01ed6-20c2-4432-a044-cf8e8a6bcb49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:33.895707 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e01ed6-20c2-4432-a044-cf8e8a6bcb49 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:33.913409 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 728/1467] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:36:36.309600 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240cb382-2b04-4781-b63e-c1e9d8dcecbe None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:36.310505 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240cb382-2b04-4781-b63e-c1e9d8dcecbe None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:36.310696 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240cb382-2b04-4781-b63e-c1e9d8dcecbe None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:36.313230 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 740/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:36:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:36.321296 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e82a2-018b-4078-9c93-7420cebe985d None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:36.321856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e82a2-018b-4078-9c93-7420cebe985d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:36.322020 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e82a2-018b-4078-9c93-7420cebe985d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:36.349006 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-b02e82a2-018b-4078-9c93-7420cebe985d None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:36.351156 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b02e82a2-018b-4078-9c93-7420cebe985d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:36.364254 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 729/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:36:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:36:36.403428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240cb382-2b04-4781-b63e-c1e9d8dcecbe None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:36.421493 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240cb382-2b04-4781-b63e-c1e9d8dcecbe None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:36.427443 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c76f5e-25bb-418a-a11b-70f4b5cb17a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:36.428501 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c76f5e-25bb-418a-a11b-70f4b5cb17a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:36.428998 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c76f5e-25bb-418a-a11b-70f4b5cb17a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:36.429541 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c76f5e-25bb-418a-a11b-70f4b5cb17a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:36.447120 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c76f5e-25bb-418a-a11b-70f4b5cb17a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:36.449859 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c76f5e-25bb-418a-a11b-70f4b5cb17a1 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:36.467939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 741/1470] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:36:46.340693 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=26 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:46.459788 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=26 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:57.786423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0424f2-6415-4ba1-a63c-9db040ce4a52 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:57.787220 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0424f2-6415-4ba1-a63c-9db040ce4a52 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:57.787220 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0424f2-6415-4ba1-a63c-9db040ce4a52 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:57.788576 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 730/1471] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:57.795222 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c962857-984b-49a2-aee1-e6b63ef1ea8b None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:57.797034 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c962857-984b-49a2-aee1-e6b63ef1ea8b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:57.797034 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c962857-984b-49a2-aee1-e6b63ef1ea8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:57.815060 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-0c962857-984b-49a2-aee1-e6b63ef1ea8b None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:57.816839 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-0c962857-984b-49a2-aee1-e6b63ef1ea8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:57.834305 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 742/1472] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:36:57.885063 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd987480-491e-41fd-ac6b-55da48956613 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:57.885325 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd987480-491e-41fd-ac6b-55da48956613 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:57.885607 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd987480-491e-41fd-ac6b-55da48956613 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:57.887095 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 731/1473] 10.222.0.45 () {54 vars in 962 bytes} [Wed Jun 15 14:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:57.896410 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9299d82-e515-4a00-adad-62c5198f79a9 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:57.896888 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9299d82-e515-4a00-adad-62c5198f79a9 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:57.897359 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9299d82-e515-4a00-adad-62c5198f79a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:57.925190 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-c9299d82-e515-4a00-adad-62c5198f79a9 None None] MFA Rules not processed for user `e4f10214b2bb4eafb3dff7adf71b9bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:57.928813 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c9299d82-e515-4a00-adad-62c5198f79a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:57.937728 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 743/1474] 10.222.0.45 () {58 vars in 1059 bytes} [Wed Jun 15 14:36:57 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:36:57.946337 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd987480-491e-41fd-ac6b-55da48956613 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:57.962261 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd987480-491e-41fd-ac6b-55da48956613 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:57.966140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85616a2-784f-4fa3-b8f2-ee80cb2530e0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f10214b2bb4eafb3dff7adf71b9bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:57.966790 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85616a2-784f-4fa3-b8f2-ee80cb2530e0 service glance] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:57.967106 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85616a2-784f-4fa3-b8f2-ee80cb2530e0 service glance] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:57.967380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85616a2-784f-4fa3-b8f2-ee80cb2530e0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:57.977487 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85616a2-784f-4fa3-b8f2-ee80cb2530e0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:57.979074 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85616a2-784f-4fa3-b8f2-ee80cb2530e0 service glance] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:57.990460 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 732/1475] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:37:00.438952 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387d3c1-98a4-4b48-93c1-2f7dab434398 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:00.439611 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387d3c1-98a4-4b48-93c1-2f7dab434398 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:00.439976 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387d3c1-98a4-4b48-93c1-2f7dab434398 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:00.442272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 744/1476] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:37:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:37:00.448262 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f20c2e-81b2-4f72-a693-33586536fe77 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:00.448749 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f20c2e-81b2-4f72-a693-33586536fe77 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:00.448995 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f20c2e-81b2-4f72-a693-33586536fe77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:00.467754 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-d9f20c2e-81b2-4f72-a693-33586536fe77 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:37:00.468968 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d9f20c2e-81b2-4f72-a693-33586536fe77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:37:00.477388 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 733/1477] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:37:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:37:00.509027 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6387d3c1-98a4-4b48-93c1-2f7dab434398 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:00.527320 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6387d3c1-98a4-4b48-93c1-2f7dab434398 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:00.541252 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7b267-23e7-4550-b21d-18bc232fd4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:00.541252 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7b267-23e7-4550-b21d-18bc232fd4d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:00.541252 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7b267-23e7-4550-b21d-18bc232fd4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:00.541252 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7b267-23e7-4550-b21d-18bc232fd4d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:00.541762 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df7b267-23e7-4550-b21d-18bc232fd4d0 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:00.541762 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: WARNING py.warnings [None req-9df7b267-23e7-4550-b21d-18bc232fd4d0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:37:00.541762 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: warnings.warn(msg) Jun 15 14:37:00.541762 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: Jun 15 14:37:00.541762 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df7b267-23e7-4550-b21d-18bc232fd4d0 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:00.568290 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 745/1478] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Jun 15 14:37:00 2022] GET /identity/v3/projects => generated 2553 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:37:02.958159 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e4b48-b64d-43f2-88cb-3a9d8b1dd211 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:02.958762 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e4b48-b64d-43f2-88cb-3a9d8b1dd211 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:02.959173 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e4b48-b64d-43f2-88cb-3a9d8b1dd211 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:02.961522 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 734/1479] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:37:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:37:02.967928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5d915-d92d-45d4-a432-0c0056a6d0d6 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:02.968440 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5d915-d92d-45d4-a432-0c0056a6d0d6 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:02.968549 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5d915-d92d-45d4-a432-0c0056a6d0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:02.992647 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-8bf5d915-d92d-45d4-a432-0c0056a6d0d6 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:37:02.994149 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-8bf5d915-d92d-45d4-a432-0c0056a6d0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:37:03.005604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 746/1480] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:37:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:37:03.063170 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3e4b48-b64d-43f2-88cb-3a9d8b1dd211 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:03.078794 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3e4b48-b64d-43f2-88cb-3a9d8b1dd211 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:03.084921 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c392f97-f47a-4d16-8e73-a10f84ee0048 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:03.085588 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c392f97-f47a-4d16-8e73-a10f84ee0048 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:03.085870 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c392f97-f47a-4d16-8e73-a10f84ee0048 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:03.086147 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c392f97-f47a-4d16-8e73-a10f84ee0048 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:03.095723 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c392f97-f47a-4d16-8e73-a10f84ee0048 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:03.097868 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c392f97-f47a-4d16-8e73-a10f84ee0048 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:03.108261 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 735/1481] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:37:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:37:05.655336 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf504c-4d2f-4aeb-b8c5-9693e71e4898 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:05.656217 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf504c-4d2f-4aeb-b8c5-9693e71e4898 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:05.656217 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf504c-4d2f-4aeb-b8c5-9693e71e4898 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:05.657294 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 747/1482] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:37:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:37:05.664754 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2959701-f8c7-4b97-83e6-b444b120a976 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:05.665572 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2959701-f8c7-4b97-83e6-b444b120a976 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:05.665757 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2959701-f8c7-4b97-83e6-b444b120a976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:05.688350 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-e2959701-f8c7-4b97-83e6-b444b120a976 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:37:05.689601 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e2959701-f8c7-4b97-83e6-b444b120a976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:37:05.697796 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 736/1483] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:37:05.735979 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf504c-4d2f-4aeb-b8c5-9693e71e4898 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:05.748563 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf504c-4d2f-4aeb-b8c5-9693e71e4898 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:05.753346 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c41c2-dcab-4cd3-8a8e-b906adfdfc24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:05.754191 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c41c2-dcab-4cd3-8a8e-b906adfdfc24 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:05.754692 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c41c2-dcab-4cd3-8a8e-b906adfdfc24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:05.755153 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c41c2-dcab-4cd3-8a8e-b906adfdfc24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:05.765741 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2c41c2-dcab-4cd3-8a8e-b906adfdfc24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:05.767219 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2c41c2-dcab-4cd3-8a8e-b906adfdfc24 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:05.779624 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 748/1484] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:37:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:37:08.183538 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0611c-f0cf-4ce4-984c-4768329e5895 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:08.184212 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0611c-f0cf-4ce4-984c-4768329e5895 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:08.184212 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0611c-f0cf-4ce4-984c-4768329e5895 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:08.185502 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 737/1485] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:37:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:37:08.192046 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3722bf7-1674-4d5f-aa26-7094a610cf5a None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:08.192540 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3722bf7-1674-4d5f-aa26-7094a610cf5a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:08.192800 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3722bf7-1674-4d5f-aa26-7094a610cf5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:08.217530 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-e3722bf7-1674-4d5f-aa26-7094a610cf5a None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:37:08.218741 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e3722bf7-1674-4d5f-aa26-7094a610cf5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:37:08.227083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 749/1486] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:37:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:37:08.265740 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0611c-f0cf-4ce4-984c-4768329e5895 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:08.290510 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0611c-f0cf-4ce4-984c-4768329e5895 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:08.294303 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d362435-507b-4f35-a38a-fa265fa82c02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:08.294950 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d362435-507b-4f35-a38a-fa265fa82c02 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:08.295359 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d362435-507b-4f35-a38a-fa265fa82c02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:08.295916 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d362435-507b-4f35-a38a-fa265fa82c02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:08.311505 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d362435-507b-4f35-a38a-fa265fa82c02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:08.313235 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d362435-507b-4f35-a38a-fa265fa82c02 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:08.324653 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 738/1487] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:37:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:37:10.686301 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f228d6-2b9a-4bca-bbb4-6b06ecd94652 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:10.686961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f228d6-2b9a-4bca-bbb4-6b06ecd94652 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:10.687149 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f228d6-2b9a-4bca-bbb4-6b06ecd94652 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:10.688857 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 750/1488] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:37:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:37:10.695152 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f05166-bbee-45e6-9a7a-47bcd8e63763 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:10.695777 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f05166-bbee-45e6-9a7a-47bcd8e63763 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:10.695777 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f05166-bbee-45e6-9a7a-47bcd8e63763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:10.719248 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-e3f05166-bbee-45e6-9a7a-47bcd8e63763 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:37:10.720467 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e3f05166-bbee-45e6-9a7a-47bcd8e63763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:37:10.733182 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 739/1489] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:37:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:37:10.778616 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f228d6-2b9a-4bca-bbb4-6b06ecd94652 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:10.792934 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f228d6-2b9a-4bca-bbb4-6b06ecd94652 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:10.797649 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e94217-0e41-4a0a-96fc-b772b5984b42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:10.798298 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e94217-0e41-4a0a-96fc-b772b5984b42 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:10.798589 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e94217-0e41-4a0a-96fc-b772b5984b42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:10.798864 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e94217-0e41-4a0a-96fc-b772b5984b42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:10.808912 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e94217-0e41-4a0a-96fc-b772b5984b42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:10.810383 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e94217-0e41-4a0a-96fc-b772b5984b42 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:10.821345 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 751/1490] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:37:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:37:13.432264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cfe27-8741-49ab-b72c-88a43e22e2f6 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:13.432817 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cfe27-8741-49ab-b72c-88a43e22e2f6 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:13.432938 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cfe27-8741-49ab-b72c-88a43e22e2f6 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:13.434570 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 740/1491] 10.222.0.45 () {54 vars in 926 bytes} [Wed Jun 15 14:37:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:37:13.440593 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe30f2-0f99-4fe3-a951-81927ff202ab None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:13.441147 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe30f2-0f99-4fe3-a951-81927ff202ab None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:13.441258 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe30f2-0f99-4fe3-a951-81927ff202ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:13.465904 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-6abe30f2-0f99-4fe3-a951-81927ff202ab None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:37:13.467129 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6abe30f2-0f99-4fe3-a951-81927ff202ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:37:13.476436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 752/1492] 10.222.0.45 () {58 vars in 1023 bytes} [Wed Jun 15 14:37:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:37:13.517285 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27cfe27-8741-49ab-b72c-88a43e22e2f6 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:13.538235 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27cfe27-8741-49ab-b72c-88a43e22e2f6 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:13.542127 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c6d3b4-d607-441e-809e-0bd64e22cf45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:13.542781 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6d3b4-d607-441e-809e-0bd64e22cf45 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:13.543229 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6d3b4-d607-441e-809e-0bd64e22cf45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:13.543556 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6d3b4-d607-441e-809e-0bd64e22cf45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:13.554625 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c6d3b4-d607-441e-809e-0bd64e22cf45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:13.556141 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c6d3b4-d607-441e-809e-0bd64e22cf45 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:13.567737 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 741/1493] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:37:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:37:23.460169 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=33 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:23.562012 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=33 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:38:46.422613 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a15442-d42c-4d28-8d45-76ea3e778632 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:46.423290 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a15442-d42c-4d28-8d45-76ea3e778632 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:46.423290 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a15442-d42c-4d28-8d45-76ea3e778632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:46.442843 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-53a15442-d42c-4d28-8d45-76ea3e778632 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:38:46.444094 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-53a15442-d42c-4d28-8d45-76ea3e778632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:38:46.452306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 753/1494] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:38:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:38:46.462724 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e6495-7adf-453d-9d53-8e18abf1f83d None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:46.463548 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e6495-7adf-453d-9d53-8e18abf1f83d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:46.463548 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e6495-7adf-453d-9d53-8e18abf1f83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:46.486100 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-670e6495-7adf-453d-9d53-8e18abf1f83d None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:38:46.487573 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-670e6495-7adf-453d-9d53-8e18abf1f83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:38:46.495663 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 742/1495] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:38:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:38:46.501346 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a15442-d42c-4d28-8d45-76ea3e778632 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:46.512243 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a15442-d42c-4d28-8d45-76ea3e778632 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:46.518263 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77334a63-944c-4e6c-8e6f-b506373e5732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:46.519135 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77334a63-944c-4e6c-8e6f-b506373e5732 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:46.519468 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77334a63-944c-4e6c-8e6f-b506373e5732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:46.519891 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77334a63-944c-4e6c-8e6f-b506373e5732 admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:46.520774 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77334a63-944c-4e6c-8e6f-b506373e5732 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:38:46.521124 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77334a63-944c-4e6c-8e6f-b506373e5732 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:46.522664 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: WARNING py.warnings [None req-77334a63-944c-4e6c-8e6f-b506373e5732 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:38:46.522664 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: warnings.warn(msg) Jun 15 14:38:46.522664 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: Jun 15 14:38:46.524222 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77334a63-944c-4e6c-8e6f-b506373e5732 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:46.546102 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 754/1496] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:38:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:38:46.559435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e6495-7adf-453d-9d53-8e18abf1f83d None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:46.569983 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e6495-7adf-453d-9d53-8e18abf1f83d None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:46.574904 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdd87e0-c6f2-4cce-9710-aad107081423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:46.575639 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd87e0-c6f2-4cce-9710-aad107081423 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:46.575926 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd87e0-c6f2-4cce-9710-aad107081423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:46.576206 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd87e0-c6f2-4cce-9710-aad107081423 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:46.576917 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdd87e0-c6f2-4cce-9710-aad107081423 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:46.579380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: WARNING py.warnings [None req-4bdd87e0-c6f2-4cce-9710-aad107081423 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:38:46.579380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: warnings.warn(msg) Jun 15 14:38:46.579380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: Jun 15 14:38:46.580593 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdd87e0-c6f2-4cce-9710-aad107081423 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:46.637557 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 743/1497] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:38:46 2022] POST /identity/v3/projects => generated 379 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:38:46.643555 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77334a63-944c-4e6c-8e6f-b506373e5732 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:46.654412 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77334a63-944c-4e6c-8e6f-b506373e5732 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:46.727664 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b25273-3791-4cde-83a8-1f3f81488ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:46.728465 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b25273-3791-4cde-83a8-1f3f81488ec1 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:46.728846 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b25273-3791-4cde-83a8-1f3f81488ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:46.729237 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b25273-3791-4cde-83a8-1f3f81488ec1 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:46.730013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b25273-3791-4cde-83a8-1f3f81488ec1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:46.731282 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: WARNING py.warnings [None req-19b25273-3791-4cde-83a8-1f3f81488ec1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:38:46.731282 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: warnings.warn(msg) Jun 15 14:38:46.731282 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: Jun 15 14:38:46.733093 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b25273-3791-4cde-83a8-1f3f81488ec1 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:46.763232 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 755/1498] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:38:46 2022] POST /identity/v3/users => generated 290 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:38:46.768742 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdd87e0-c6f2-4cce-9710-aad107081423 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:46.785579 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdd87e0-c6f2-4cce-9710-aad107081423 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:46.791512 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610cea9-0179-4db2-8967-340ba7993747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:46.792195 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610cea9-0179-4db2-8967-340ba7993747 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:46.792479 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610cea9-0179-4db2-8967-340ba7993747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:46.792758 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610cea9-0179-4db2-8967-340ba7993747 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:46.793500 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6610cea9-0179-4db2-8967-340ba7993747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:46.794527 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: WARNING py.warnings [None req-6610cea9-0179-4db2-8967-340ba7993747 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:38:46.794527 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: warnings.warn(msg) Jun 15 14:38:46.794527 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: Jun 15 14:38:46.795464 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6610cea9-0179-4db2-8967-340ba7993747 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:46.811591 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 744/1499] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:38:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:38:46.816114 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b25273-3791-4cde-83a8-1f3f81488ec1 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:46.826948 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b25273-3791-4cde-83a8-1f3f81488ec1 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:46.831842 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c316479a-1f26-45b7-b0f0-07796e64c75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:46.832507 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316479a-1f26-45b7-b0f0-07796e64c75e admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:46.832788 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316479a-1f26-45b7-b0f0-07796e64c75e admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:46.833065 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316479a-1f26-45b7-b0f0-07796e64c75e admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:46.833816 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c316479a-1f26-45b7-b0f0-07796e64c75e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:46.834842 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: WARNING py.warnings [None req-c316479a-1f26-45b7-b0f0-07796e64c75e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:38:46.834842 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: warnings.warn(msg) Jun 15 14:38:46.834842 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: Jun 15 14:38:46.835762 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c316479a-1f26-45b7-b0f0-07796e64c75e admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:46.854313 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 756/1500] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:38:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:38:46.859575 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610cea9-0179-4db2-8967-340ba7993747 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:46.870056 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610cea9-0179-4db2-8967-340ba7993747 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:46.876407 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804aaf64-da72-47aa-84ab-93d100e316dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:46.877394 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804aaf64-da72-47aa-84ab-93d100e316dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:46.877781 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804aaf64-da72-47aa-84ab-93d100e316dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:46.878155 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804aaf64-da72-47aa-84ab-93d100e316dc admin admin] PATH_INFO: `/v3/projects/6db038a4b2044440be94df36f23e52bb/users/ccb1506b64114b2e8b221365f7b391fd/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:46.895430 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804aaf64-da72-47aa-84ab-93d100e316dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6db038a4b2044440be94df36f23e52bb, user_id=ccb1506b64114b2e8b221365f7b391fd, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:46.896546 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: WARNING py.warnings [None req-804aaf64-da72-47aa-84ab-93d100e316dc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:38:46.896546 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: warnings.warn(msg) Jun 15 14:38:46.896546 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: Jun 15 14:38:46.898978 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804aaf64-da72-47aa-84ab-93d100e316dc admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:46.917450 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 745/1501] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:38:46 2022] PUT /identity/v3/projects/6db038a4b2044440be94df36f23e52bb/users/ccb1506b64114b2e8b221365f7b391fd/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:38:46.929886 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5daf4-5d1d-4388-951d-69965324eb31 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:46.930174 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5daf4-5d1d-4388-951d-69965324eb31 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:46.930455 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5daf4-5d1d-4388-951d-69965324eb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:46.958906 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-faf5daf4-5d1d-4388-951d-69965324eb31 None None] MFA Rules not processed for user `ccb1506b64114b2e8b221365f7b391fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:38:46.960478 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-faf5daf4-5d1d-4388-951d-69965324eb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:38:47.052852 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 757/1502] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:38:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:38:47.068309 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49f5bf-9df4-4f2e-9ee7-5912b15c2858 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:47.068579 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49f5bf-9df4-4f2e-9ee7-5912b15c2858 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:47.068876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49f5bf-9df4-4f2e-9ee7-5912b15c2858 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:47.070355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 746/1503] 10.222.0.45 () {54 vars in 963 bytes} [Wed Jun 15 14:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:38:47.075487 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7fc540-7890-4a4e-be2a-a549b13f7fae None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:47.075991 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7fc540-7890-4a4e-be2a-a549b13f7fae None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:47.076488 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7fc540-7890-4a4e-be2a-a549b13f7fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:47.101648 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-1a7fc540-7890-4a4e-be2a-a549b13f7fae None None] MFA Rules not processed for user `b47cec76cb1c4afb8e011ff04a885f96`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:38:47.102864 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1a7fc540-7890-4a4e-be2a-a549b13f7fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:38:47.162904 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:38:47.171342 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 758/1504] 10.222.0.45 () {58 vars in 1060 bytes} [Wed Jun 15 14:38:47 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:38:47.181489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49f5bf-9df4-4f2e-9ee7-5912b15c2858 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:47.196427 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49f5bf-9df4-4f2e-9ee7-5912b15c2858 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:47.201103 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3123ac58-b490-4b8f-83a2-d3b2e1a1f2db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:47.201747 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123ac58-b490-4b8f-83a2-d3b2e1a1f2db service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:47.202027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123ac58-b490-4b8f-83a2-d3b2e1a1f2db service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:47.202298 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123ac58-b490-4b8f-83a2-d3b2e1a1f2db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:47.211725 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3123ac58-b490-4b8f-83a2-d3b2e1a1f2db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:47.213287 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3123ac58-b490-4b8f-83a2-d3b2e1a1f2db service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:47.225006 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 747/1505] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:38:47 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:38:47.273478 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f58659-a9a1-4a6e-9719-1e7dab6e2b9d None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:47.273757 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f58659-a9a1-4a6e-9719-1e7dab6e2b9d None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:47.274078 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f58659-a9a1-4a6e-9719-1e7dab6e2b9d None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:47.275494 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 759/1506] 10.222.0.45 () {52 vars in 829 bytes} [Wed Jun 15 14:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:38:47.279627 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3123ac58-b490-4b8f-83a2-d3b2e1a1f2db service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:47.361305 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3123ac58-b490-4b8f-83a2-d3b2e1a1f2db service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:47.365890 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017ba85-730c-4500-9493-e8dd819da222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:47.366868 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017ba85-730c-4500-9493-e8dd819da222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:47.367272 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017ba85-730c-4500-9493-e8dd819da222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:47.367637 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017ba85-730c-4500-9493-e8dd819da222 admin admin] PATH_INFO: `/v3/users/ccb1506b64114b2e8b221365f7b391fd` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:47.368878 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017ba85-730c-4500-9493-e8dd819da222 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccb1506b64114b2e8b221365f7b391fd)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:47.369900 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: WARNING py.warnings [None req-e017ba85-730c-4500-9493-e8dd819da222 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:38:47.369900 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: warnings.warn(msg) Jun 15 14:38:47.369900 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: Jun 15 14:38:47.371142 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017ba85-730c-4500-9493-e8dd819da222 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:47.482511 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e017ba85-730c-4500-9493-e8dd819da222 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccb1506b64114b2e8b221365f7b391fd'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:38:47.484253 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e017ba85-730c-4500-9493-e8dd819da222 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccb1506b64114b2e8b221365f7b391fd'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:38:47.512005 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e017ba85-730c-4500-9493-e8dd819da222 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccb1506b64114b2e8b221365f7b391fd'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:38:47.525803 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e017ba85-730c-4500-9493-e8dd819da222 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccb1506b64114b2e8b221365f7b391fd'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:38:47.543390 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:38:47.552533 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e017ba85-730c-4500-9493-e8dd819da222 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccb1506b64114b2e8b221365f7b391fd'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:38:47.560725 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 748/1507] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:38:47 2022] DELETE /identity/v3/users/ccb1506b64114b2e8b221365f7b391fd => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:38:47.565811 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d22dbd-99eb-4ed2-b5fe-bf837ee13d5d None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:47.567852 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d22dbd-99eb-4ed2-b5fe-bf837ee13d5d None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:47.567852 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d22dbd-99eb-4ed2-b5fe-bf837ee13d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:47.592262 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-67d22dbd-99eb-4ed2-b5fe-bf837ee13d5d None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:38:47.594055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-67d22dbd-99eb-4ed2-b5fe-bf837ee13d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:38:47.670968 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 760/1508] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:38:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:38:47.682199 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017ba85-730c-4500-9493-e8dd819da222 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:47.684151 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e017ba85-730c-4500-9493-e8dd819da222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:38:47.750710 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017ba85-730c-4500-9493-e8dd819da222 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:47.754625 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f672c1-a66c-42a4-82ed-a233ad10819d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:47.755323 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f672c1-a66c-42a4-82ed-a233ad10819d service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:47.755631 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f672c1-a66c-42a4-82ed-a233ad10819d service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:47.755882 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f672c1-a66c-42a4-82ed-a233ad10819d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:47.766838 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f672c1-a66c-42a4-82ed-a233ad10819d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:47.768980 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f672c1-a66c-42a4-82ed-a233ad10819d service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:47.779952 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 749/1509] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:38:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:38:48.174688 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d22dbd-99eb-4ed2-b5fe-bf837ee13d5d None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:48.176623 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-67d22dbd-99eb-4ed2-b5fe-bf837ee13d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:38:48.191845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d22dbd-99eb-4ed2-b5fe-bf837ee13d5d None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:48.197165 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e45fe5-78c7-45a9-bc00-b7e72035ce76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:48.198048 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e45fe5-78c7-45a9-bc00-b7e72035ce76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:48.198253 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e45fe5-78c7-45a9-bc00-b7e72035ce76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:48.198534 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e45fe5-78c7-45a9-bc00-b7e72035ce76 admin admin] PATH_INFO: `/v3/projects/6db038a4b2044440be94df36f23e52bb` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:48.199869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e45fe5-78c7-45a9-bc00-b7e72035ce76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6db038a4b2044440be94df36f23e52bb)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:48.200913 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: WARNING py.warnings [None req-36e45fe5-78c7-45a9-bc00-b7e72035ce76 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:38:48.200913 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: warnings.warn(msg) Jun 15 14:38:48.200913 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: Jun 15 14:38:48.202445 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e45fe5-78c7-45a9-bc00-b7e72035ce76 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:48.281271 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-36e45fe5-78c7-45a9-bc00-b7e72035ce76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6db038a4b2044440be94df36f23e52bb', 'request_id': 'req-36e45fe5-78c7-45a9-bc00-b7e72035ce76'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:38:48.292780 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-36e45fe5-78c7-45a9-bc00-b7e72035ce76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6db038a4b2044440be94df36f23e52bb', 'request_id': 'req-36e45fe5-78c7-45a9-bc00-b7e72035ce76'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:38:48.308883 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-36e45fe5-78c7-45a9-bc00-b7e72035ce76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6db038a4b2044440be94df36f23e52bb', 'request_id': 'req-36e45fe5-78c7-45a9-bc00-b7e72035ce76'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:38:48.320914 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-36e45fe5-78c7-45a9-bc00-b7e72035ce76 admin admin] The token cache is being invalidate because project 6db038a4b2044440be94df36f23e52bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:38:48.321300 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-36e45fe5-78c7-45a9-bc00-b7e72035ce76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:38:48.322181 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-36e45fe5-78c7-45a9-bc00-b7e72035ce76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:38:48.323929 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 761/1510] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:38:48 2022] DELETE /identity/v3/projects/6db038a4b2044440be94df36f23e52bb => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:38:57.775275 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=29 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:38:58.317836 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:40:35.194148 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a2db8-4270-490c-ad1b-9b0fa3901519 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:35.194737 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a2db8-4270-490c-ad1b-9b0fa3901519 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:35.194737 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a2db8-4270-490c-ad1b-9b0fa3901519 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:35.196255 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 750/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:40:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:40:35.201577 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba54223-635e-4788-ae0f-10dd111c8453 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:35.202079 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba54223-635e-4788-ae0f-10dd111c8453 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:35.202185 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba54223-635e-4788-ae0f-10dd111c8453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:35.223703 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-5ba54223-635e-4788-ae0f-10dd111c8453 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:40:35.224285 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5ba54223-635e-4788-ae0f-10dd111c8453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:40:35.292992 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 762/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:40:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:40:35.471113 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a2db8-4270-490c-ad1b-9b0fa3901519 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:40:35.473249 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-362a2db8-4270-490c-ad1b-9b0fa3901519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:40:35.571897 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a2db8-4270-490c-ad1b-9b0fa3901519 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:40:35.577479 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e174f4-01af-4b9d-bd9e-36936a831a2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:40:35.578138 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e174f4-01af-4b9d-bd9e-36936a831a2f service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:35.578417 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e174f4-01af-4b9d-bd9e-36936a831a2f service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:35.578695 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e174f4-01af-4b9d-bd9e-36936a831a2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:35.587797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e174f4-01af-4b9d-bd9e-36936a831a2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:40:35.589411 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e174f4-01af-4b9d-bd9e-36936a831a2f service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:40:35.600282 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 751/1513] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:40:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:40:36.496354 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cdfa38-2127-4d71-96c5-1a7ba9c77e9e None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:36.496971 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cdfa38-2127-4d71-96c5-1a7ba9c77e9e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:36.496971 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cdfa38-2127-4d71-96c5-1a7ba9c77e9e None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:36.498340 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 763/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:40:36.503749 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d41cd6-85a4-4b02-a134-079801f65d45 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:36.504341 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d41cd6-85a4-4b02-a134-079801f65d45 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:36.504341 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d41cd6-85a4-4b02-a134-079801f65d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:36.524316 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-41d41cd6-85a4-4b02-a134-079801f65d45 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:40:36.525525 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-41d41cd6-85a4-4b02-a134-079801f65d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:40:36.534216 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 752/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:40:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:40:36.686702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cdfa38-2127-4d71-96c5-1a7ba9c77e9e None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:40:36.704557 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cdfa38-2127-4d71-96c5-1a7ba9c77e9e None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:40:36.710245 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fa4950-478d-4269-bf68-60c0843256f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:40:36.710897 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa4950-478d-4269-bf68-60c0843256f6 service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:36.711214 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa4950-478d-4269-bf68-60c0843256f6 service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:36.711500 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa4950-478d-4269-bf68-60c0843256f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:36.720513 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fa4950-478d-4269-bf68-60c0843256f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:40:36.722143 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fa4950-478d-4269-bf68-60c0843256f6 service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:40:36.733454 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 764/1516] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:40:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:40:37.659019 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de69233-2c58-4130-80e1-85325538aaa9 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:37.659604 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de69233-2c58-4130-80e1-85325538aaa9 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:37.659604 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de69233-2c58-4130-80e1-85325538aaa9 None None] PATH_INFO: `/v3` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:37.660997 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 753/1517] 10.222.0.47 () {54 vars in 925 bytes} [Wed Jun 15 14:40:37 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:40:37.669329 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec032b42-a250-4bb5-a400-91d46b3b18fb None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:37.670652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec032b42-a250-4bb5-a400-91d46b3b18fb None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:37.670814 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec032b42-a250-4bb5-a400-91d46b3b18fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:37.693709 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-ec032b42-a250-4bb5-a400-91d46b3b18fb None None] MFA Rules not processed for user `f4128111a9aa450abb2146a5d9926dec`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:40:37.694938 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ec032b42-a250-4bb5-a400-91d46b3b18fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:40:37.767810 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 765/1518] 10.222.0.47 () {58 vars in 1015 bytes} [Wed Jun 15 14:40:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:40:46.518721 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=37 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:40:47.758300 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=62 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:40:47.907227 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb0e44-2029-4f06-b1fa-764532bd966d None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:47.907762 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb0e44-2029-4f06-b1fa-764532bd966d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:47.907762 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb0e44-2029-4f06-b1fa-764532bd966d None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:47.909334 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 754/1519] 10.222.0.45 () {54 vars in 960 bytes} [Wed Jun 15 14:40:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:40:47.914614 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4775a1-72a5-47f1-9a4c-0bec20077656 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:47.914939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4775a1-72a5-47f1-9a4c-0bec20077656 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:47.916005 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4775a1-72a5-47f1-9a4c-0bec20077656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:47.934194 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-af4775a1-72a5-47f1-9a4c-0bec20077656 None None] MFA Rules not processed for user `f4128111a9aa450abb2146a5d9926dec`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:40:47.935944 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-af4775a1-72a5-47f1-9a4c-0bec20077656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:40:47.945102 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 766/1520] 10.222.0.45 () {58 vars in 1057 bytes} [Wed Jun 15 14:40:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:40:47.953349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eb0e44-2029-4f06-b1fa-764532bd966d None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:40:47.967210 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eb0e44-2029-4f06-b1fa-764532bd966d None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:40:47.972438 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43469b7-6a9f-432e-981b-c3db2e99d394 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4128111a9aa450abb2146a5d9926dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:40:47.973083 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43469b7-6a9f-432e-981b-c3db2e99d394 service placement] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:47.973496 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43469b7-6a9f-432e-981b-c3db2e99d394 service placement] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:47.973784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43469b7-6a9f-432e-981b-c3db2e99d394 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:47.982823 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43469b7-6a9f-432e-981b-c3db2e99d394 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:40:47.984503 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43469b7-6a9f-432e-981b-c3db2e99d394 service placement] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:40:47.994625 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 755/1521] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:40:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:40:48.058065 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee0297-38fb-4b5d-8b77-936d78b6bad1 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:48.058359 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee0297-38fb-4b5d-8b77-936d78b6bad1 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:48.058645 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee0297-38fb-4b5d-8b77-936d78b6bad1 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:48.060148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 767/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:40:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:40:48.065240 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc923050-8345-4d49-b4f9-11dc102cd277 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:48.065529 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc923050-8345-4d49-b4f9-11dc102cd277 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:48.065803 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc923050-8345-4d49-b4f9-11dc102cd277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:48.084484 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-fc923050-8345-4d49-b4f9-11dc102cd277 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:40:48.085712 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-fc923050-8345-4d49-b4f9-11dc102cd277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:40:48.094103 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 756/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:40:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:40:48.259968 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee0297-38fb-4b5d-8b77-936d78b6bad1 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:40:48.277496 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee0297-38fb-4b5d-8b77-936d78b6bad1 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:40:48.284892 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f86997-da1b-413b-b54f-c0d9f233d587 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:40:48.285552 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f86997-da1b-413b-b54f-c0d9f233d587 service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:48.285838 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f86997-da1b-413b-b54f-c0d9f233d587 service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:48.286116 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f86997-da1b-413b-b54f-c0d9f233d587 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:48.295705 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f86997-da1b-413b-b54f-c0d9f233d587 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:40:48.298653 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f86997-da1b-413b-b54f-c0d9f233d587 service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:40:48.307266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 768/1524] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:40:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:40:49.235598 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fcf029-eb0c-48bf-9000-c2bba6280dc0 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:49.236177 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fcf029-eb0c-48bf-9000-c2bba6280dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:49.236177 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fcf029-eb0c-48bf-9000-c2bba6280dc0 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:49.237543 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 757/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:40:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:40:49.242455 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe04228-e9e3-4bc5-98f0-12b0d3cbfbc4 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:49.242892 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe04228-e9e3-4bc5-98f0-12b0d3cbfbc4 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:49.243031 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe04228-e9e3-4bc5-98f0-12b0d3cbfbc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:49.261195 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-2fe04228-e9e3-4bc5-98f0-12b0d3cbfbc4 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:40:49.262393 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-2fe04228-e9e3-4bc5-98f0-12b0d3cbfbc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:40:49.270737 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 769/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:40:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:40:49.447485 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fcf029-eb0c-48bf-9000-c2bba6280dc0 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:40:49.462442 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fcf029-eb0c-48bf-9000-c2bba6280dc0 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:40:49.468065 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa7a70-5866-41b8-8382-987b1ed7c4fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:40:49.468713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa7a70-5866-41b8-8382-987b1ed7c4fd service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:49.469052 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa7a70-5866-41b8-8382-987b1ed7c4fd service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:49.469276 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa7a70-5866-41b8-8382-987b1ed7c4fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:49.478212 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa7a70-5866-41b8-8382-987b1ed7c4fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:40:49.479865 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa7a70-5866-41b8-8382-987b1ed7c4fd service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:40:49.489820 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 758/1527] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:40:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:40:53.670260 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ee073-0c2c-4b39-9b37-17eca1c9775a None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:53.670801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ee073-0c2c-4b39-9b37-17eca1c9775a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:53.670801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ee073-0c2c-4b39-9b37-17eca1c9775a None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:53.672223 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 770/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:40:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:40:53.678146 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4606d-744b-4f88-80ee-b9ed213a1686 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:53.678615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4606d-744b-4f88-80ee-b9ed213a1686 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:53.678722 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4606d-744b-4f88-80ee-b9ed213a1686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:53.697618 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-d0f4606d-744b-4f88-80ee-b9ed213a1686 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:40:53.698828 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d0f4606d-744b-4f88-80ee-b9ed213a1686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:40:53.707351 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 759/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:40:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:40:53.746487 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ee073-0c2c-4b39-9b37-17eca1c9775a None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:40:53.748823 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-302ee073-0c2c-4b39-9b37-17eca1c9775a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:40:53.819157 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ee073-0c2c-4b39-9b37-17eca1c9775a None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:40:53.823166 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5932dc21-6f0b-4ee5-a7a8-e7df03420bbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:40:53.824064 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932dc21-6f0b-4ee5-a7a8-e7df03420bbb service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:53.824210 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932dc21-6f0b-4ee5-a7a8-e7df03420bbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:53.824440 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932dc21-6f0b-4ee5-a7a8-e7df03420bbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:53.834326 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5932dc21-6f0b-4ee5-a7a8-e7df03420bbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:40:53.835816 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5932dc21-6f0b-4ee5-a7a8-e7df03420bbb service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:40:53.846582 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 771/1530] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:40:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:40:56.077423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce0941-1d58-4bbc-946c-eaf4a9cf0077 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:56.078234 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce0941-1d58-4bbc-946c-eaf4a9cf0077 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:56.078591 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce0941-1d58-4bbc-946c-eaf4a9cf0077 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:56.081305 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 760/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:40:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:40:56.087449 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cf3e1-626f-480f-9948-d510d272af82 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:56.087969 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cf3e1-626f-480f-9948-d510d272af82 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:56.088064 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cf3e1-626f-480f-9948-d510d272af82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:56.108173 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-704cf3e1-626f-480f-9948-d510d272af82 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:40:56.113050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-704cf3e1-626f-480f-9948-d510d272af82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:40:56.117704 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 772/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:40:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:40:56.155490 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce0941-1d58-4bbc-946c-eaf4a9cf0077 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:40:56.168243 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce0941-1d58-4bbc-946c-eaf4a9cf0077 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:40:56.174702 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad8ea9-e279-4671-a51b-b48892aec7b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:40:56.175795 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad8ea9-e279-4671-a51b-b48892aec7b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:56.176260 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad8ea9-e279-4671-a51b-b48892aec7b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:56.176707 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad8ea9-e279-4671-a51b-b48892aec7b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:56.192339 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad8ea9-e279-4671-a51b-b48892aec7b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:40:56.194900 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad8ea9-e279-4671-a51b-b48892aec7b4 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:40:56.210150 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 761/1533] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:40:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:41:00.249152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab363ea0-4adc-4250-b5de-8819e1faf91c None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:00.249699 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab363ea0-4adc-4250-b5de-8819e1faf91c None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:00.249841 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab363ea0-4adc-4250-b5de-8819e1faf91c None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:00.251247 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 773/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:41:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:41:00.256822 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f55b3-027c-40d5-9d3e-9b36141f4652 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:00.257317 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f55b3-027c-40d5-9d3e-9b36141f4652 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:00.257441 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f55b3-027c-40d5-9d3e-9b36141f4652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:00.276773 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-8b9f55b3-027c-40d5-9d3e-9b36141f4652 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:41:00.278012 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-8b9f55b3-027c-40d5-9d3e-9b36141f4652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:00.286592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 762/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:41:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:41:00.558107 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab363ea0-4adc-4250-b5de-8819e1faf91c None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:41:00.560398 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ab363ea0-4adc-4250-b5de-8819e1faf91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:00.625887 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab363ea0-4adc-4250-b5de-8819e1faf91c None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:41:00.629453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9edfc2-eca6-4279-8278-183b23fe570e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:41:00.630116 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9edfc2-eca6-4279-8278-183b23fe570e service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:00.630396 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9edfc2-eca6-4279-8278-183b23fe570e service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:00.630676 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9edfc2-eca6-4279-8278-183b23fe570e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:00.640998 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9edfc2-eca6-4279-8278-183b23fe570e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:41:00.642477 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9edfc2-eca6-4279-8278-183b23fe570e service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:41:00.653133 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 774/1536] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:41:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:41:02.923058 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cf878-3ba9-4f0d-879e-de75e659c565 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:02.923593 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cf878-3ba9-4f0d-879e-de75e659c565 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:02.923798 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cf878-3ba9-4f0d-879e-de75e659c565 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:02.925796 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 763/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:41:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:41:02.931427 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f4f9d-6045-489f-a21e-4e0cb8419a29 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:02.931895 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f4f9d-6045-489f-a21e-4e0cb8419a29 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:02.932040 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f4f9d-6045-489f-a21e-4e0cb8419a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:02.954075 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-468f4f9d-6045-489f-a21e-4e0cb8419a29 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:41:02.956073 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-468f4f9d-6045-489f-a21e-4e0cb8419a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:02.966375 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 775/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:41:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:41:03.275504 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27cf878-3ba9-4f0d-879e-de75e659c565 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:41:03.277258 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-a27cf878-3ba9-4f0d-879e-de75e659c565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:03.299645 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27cf878-3ba9-4f0d-879e-de75e659c565 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:41:03.303182 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ace2e-0ea7-4a67-91be-8c8f8cd2ac72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:41:03.303838 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ace2e-0ea7-4a67-91be-8c8f8cd2ac72 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:03.304122 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ace2e-0ea7-4a67-91be-8c8f8cd2ac72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:03.304396 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ace2e-0ea7-4a67-91be-8c8f8cd2ac72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:03.313807 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615ace2e-0ea7-4a67-91be-8c8f8cd2ac72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:41:03.315287 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615ace2e-0ea7-4a67-91be-8c8f8cd2ac72 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:41:03.326013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 764/1539] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:41:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:41:12.944661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=70 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:41:13.320759 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=33 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:42:40.596675 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17c8f6-6613-4af7-8c53-0eeb0233bc7e None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:40.597787 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17c8f6-6613-4af7-8c53-0eeb0233bc7e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:40.598278 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17c8f6-6613-4af7-8c53-0eeb0233bc7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:40.626383 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-9e17c8f6-6613-4af7-8c53-0eeb0233bc7e None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:40.628106 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-9e17c8f6-6613-4af7-8c53-0eeb0233bc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:40.646155 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 776/1540] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:40.660962 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b6138-064d-4024-a13d-94d263c289e2 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:40.661633 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b6138-064d-4024-a13d-94d263c289e2 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:40.661633 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b6138-064d-4024-a13d-94d263c289e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:40.666662 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b2195-1ff4-4546-89dc-32cdfad065e3 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:40.667324 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b2195-1ff4-4546-89dc-32cdfad065e3 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:40.667833 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b2195-1ff4-4546-89dc-32cdfad065e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:40.696185 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-d40b6138-064d-4024-a13d-94d263c289e2 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:40.697463 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d40b6138-064d-4024-a13d-94d263c289e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:40.704312 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-575b2195-1ff4-4546-89dc-32cdfad065e3 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:40.709247 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-575b2195-1ff4-4546-89dc-32cdfad065e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:40.716128 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 765/1541] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:40.721514 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b6138-064d-4024-a13d-94d263c289e2 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:40.731850 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 777/1542] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:42:40.738497 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6040a-b0fe-4459-ae12-807978a9a13f None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:40.739672 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6040a-b0fe-4459-ae12-807978a9a13f None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:40.740960 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6040a-b0fe-4459-ae12-807978a9a13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:40.741450 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b6138-064d-4024-a13d-94d263c289e2 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:40.752909 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594703f2-a6b3-4971-9872-ec2a160f2eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:40.753671 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594703f2-a6b3-4971-9872-ec2a160f2eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:40.753967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594703f2-a6b3-4971-9872-ec2a160f2eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:40.754246 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594703f2-a6b3-4971-9872-ec2a160f2eb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:40.754987 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594703f2-a6b3-4971-9872-ec2a160f2eb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:42:40.755326 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594703f2-a6b3-4971-9872-ec2a160f2eb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:40.757023 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: WARNING py.warnings [None req-594703f2-a6b3-4971-9872-ec2a160f2eb2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:42:40.757023 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: warnings.warn(msg) Jun 15 14:42:40.757023 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: Jun 15 14:42:40.758122 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594703f2-a6b3-4971-9872-ec2a160f2eb2 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:40.770425 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-a0b6040a-b0fe-4459-ae12-807978a9a13f None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:40.776493 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 766/1543] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:42:40.777666 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a0b6040a-b0fe-4459-ae12-807978a9a13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:40.778166 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c4d281-cbec-45b5-b963-8438b8e11cde None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:40.778468 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c4d281-cbec-45b5-b963-8438b8e11cde None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:40.778765 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c4d281-cbec-45b5-b963-8438b8e11cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:40.808775 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 778/1544] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:42:40.810906 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d6ded-3173-4b5c-9451-636740f477bb None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:40.811458 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d6ded-3173-4b5c-9451-636740f477bb None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:40.811929 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d6ded-3173-4b5c-9451-636740f477bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:40.826746 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-43c4d281-cbec-45b5-b963-8438b8e11cde None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:40.831242 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-43c4d281-cbec-45b5-b963-8438b8e11cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:40.844930 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 767/1545] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:40.848450 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523e21d-903d-4f77-a9cb-bde1189ce7ca None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:40.849011 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523e21d-903d-4f77-a9cb-bde1189ce7ca None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:40.849526 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523e21d-903d-4f77-a9cb-bde1189ce7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:40.862208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-976d6ded-3173-4b5c-9451-636740f477bb None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:40.864334 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-976d6ded-3173-4b5c-9451-636740f477bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:40.873471 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-e523e21d-903d-4f77-a9cb-bde1189ce7ca None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:40.875121 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e523e21d-903d-4f77-a9cb-bde1189ce7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:40.882249 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 779/1546] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:40.883834 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d6ded-3173-4b5c-9451-636740f477bb None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:40.885621 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 768/1547] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:40.887177 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a1e9d-4d39-49fc-960d-39b57df93910 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:40.887642 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a1e9d-4d39-49fc-960d-39b57df93910 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:40.888228 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a1e9d-4d39-49fc-960d-39b57df93910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:40.910059 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-5d3a1e9d-4d39-49fc-960d-39b57df93910 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:40.911620 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-5d3a1e9d-4d39-49fc-960d-39b57df93910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:40.913077 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d6ded-3173-4b5c-9451-636740f477bb None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:40.922055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e5a14f-74b2-4463-8c8f-c96d4110715c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:40.922483 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 769/1548] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:40.922660 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5a14f-74b2-4463-8c8f-c96d4110715c admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:40.923133 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5a14f-74b2-4463-8c8f-c96d4110715c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:40.923483 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5a14f-74b2-4463-8c8f-c96d4110715c admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:40.924208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e5a14f-74b2-4463-8c8f-c96d4110715c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:40.924909 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cdfec5-d100-45ad-a4dd-2ce4b9fdd727 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:40.925479 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cdfec5-d100-45ad-a4dd-2ce4b9fdd727 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:40.925607 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: WARNING py.warnings [None req-a5e5a14f-74b2-4463-8c8f-c96d4110715c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:42:40.925607 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: warnings.warn(msg) Jun 15 14:42:40.925607 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: Jun 15 14:42:40.926293 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cdfec5-d100-45ad-a4dd-2ce4b9fdd727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:40.927031 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e5a14f-74b2-4463-8c8f-c96d4110715c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:40.948911 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-86cdfec5-d100-45ad-a4dd-2ce4b9fdd727 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:40.950185 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-86cdfec5-d100-45ad-a4dd-2ce4b9fdd727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:40.958665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 770/1549] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:40.960653 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918316c-c9d8-462f-9bac-5509d44124e8 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:40.960944 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918316c-c9d8-462f-9bac-5509d44124e8 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:40.961224 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918316c-c9d8-462f-9bac-5509d44124e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:40.969622 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 780/1550] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:42:40 2022] POST /identity/v3/projects => generated 393 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:42:40.970587 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e5a14f-74b2-4463-8c8f-c96d4110715c admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:40.982584 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-6918316c-c9d8-462f-9bac-5509d44124e8 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:40.984270 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-6918316c-c9d8-462f-9bac-5509d44124e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:41.047701 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e5a14f-74b2-4463-8c8f-c96d4110715c admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:41.052698 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0851489-cc00-4803-922e-68ddd29edd79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:41.053377 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0851489-cc00-4803-922e-68ddd29edd79 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:41.053665 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0851489-cc00-4803-922e-68ddd29edd79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:41.053944 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0851489-cc00-4803-922e-68ddd29edd79 admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:41.054683 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0851489-cc00-4803-922e-68ddd29edd79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:42:41.055019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0851489-cc00-4803-922e-68ddd29edd79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:41.056686 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0851489-cc00-4803-922e-68ddd29edd79 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:41.065549 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 771/1551] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:41.066870 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47d9e0-f43a-4e4e-9eb4-3fddf821138a None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:41.067409 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47d9e0-f43a-4e4e-9eb4-3fddf821138a None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:41.067695 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47d9e0-f43a-4e4e-9eb4-3fddf821138a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:41.070292 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 781/1552] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:42:41.071781 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a45bf-b8fe-44fa-89b6-bff5c63d197e None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:41.072079 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a45bf-b8fe-44fa-89b6-bff5c63d197e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:41.072363 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a45bf-b8fe-44fa-89b6-bff5c63d197e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:41.090361 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-ba47d9e0-f43a-4e4e-9eb4-3fddf821138a None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:41.091490 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ba47d9e0-f43a-4e4e-9eb4-3fddf821138a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:41.092977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-ae0a45bf-b8fe-44fa-89b6-bff5c63d197e None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:41.094304 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ae0a45bf-b8fe-44fa-89b6-bff5c63d197e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:41.100352 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 772/1553] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:41.101560 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb00734-00da-490b-be90-9adc809f969f None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:41.101850 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb00734-00da-490b-be90-9adc809f969f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:41.102152 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb00734-00da-490b-be90-9adc809f969f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:41.103516 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 782/1554] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:41.108203 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b59389-6d20-43a3-9d28-d57b58da8f23 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:41.108968 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b59389-6d20-43a3-9d28-d57b58da8f23 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:41.108968 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b59389-6d20-43a3-9d28-d57b58da8f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:41.121766 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-ceb00734-00da-490b-be90-9adc809f969f None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:41.122978 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ceb00734-00da-490b-be90-9adc809f969f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:41.131725 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 773/1555] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:41.132607 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb00734-00da-490b-be90-9adc809f969f None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:41.135684 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-17b59389-6d20-43a3-9d28-d57b58da8f23 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:41.137232 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-17b59389-6d20-43a3-9d28-d57b58da8f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:41.147675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb00734-00da-490b-be90-9adc809f969f None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:41.148458 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 783/1556] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:41.151311 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b59389-6d20-43a3-9d28-d57b58da8f23 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:41.160405 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736c73b0-8b95-4cc9-a002-6556e2f23bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:41.161428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b59389-6d20-43a3-9d28-d57b58da8f23 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:41.161701 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c73b0-8b95-4cc9-a002-6556e2f23bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:41.162724 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c73b0-8b95-4cc9-a002-6556e2f23bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:41.163226 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c73b0-8b95-4cc9-a002-6556e2f23bb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:41.164206 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736c73b0-8b95-4cc9-a002-6556e2f23bb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:42:41.164676 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736c73b0-8b95-4cc9-a002-6556e2f23bb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:41.166448 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736c73b0-8b95-4cc9-a002-6556e2f23bb2 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:41.166636 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a3050-f82c-4017-9783-471a656895ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:41.167328 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a3050-f82c-4017-9783-471a656895ff admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:41.167609 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a3050-f82c-4017-9783-471a656895ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:41.167882 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a3050-f82c-4017-9783-471a656895ff admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:41.168566 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3a3050-f82c-4017-9783-471a656895ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:41.170468 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3a3050-f82c-4017-9783-471a656895ff admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:41.180703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 774/1557] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:42:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:42:41.181557 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736c73b0-8b95-4cc9-a002-6556e2f23bb2 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:41.194114 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736c73b0-8b95-4cc9-a002-6556e2f23bb2 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:41.197531 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 784/1558] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:42:41 2022] POST /identity/v3/users => generated 297 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:42:41.198631 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a3050-f82c-4017-9783-471a656895ff admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:41.198806 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ceecaa-aad5-4e4d-be02-c8b008f32ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:41.199648 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ceecaa-aad5-4e4d-be02-c8b008f32ac7 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:41.199936 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ceecaa-aad5-4e4d-be02-c8b008f32ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:41.200213 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ceecaa-aad5-4e4d-be02-c8b008f32ac7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:41.201013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ceecaa-aad5-4e4d-be02-c8b008f32ac7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:42:41.201318 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ceecaa-aad5-4e4d-be02-c8b008f32ac7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:41.203465 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ceecaa-aad5-4e4d-be02-c8b008f32ac7 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:41.211433 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a3050-f82c-4017-9783-471a656895ff admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:41.216415 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293955af-e88e-4556-aa88-c33f1df0a855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:41.217089 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293955af-e88e-4556-aa88-c33f1df0a855 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:41.217375 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293955af-e88e-4556-aa88-c33f1df0a855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:41.217653 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293955af-e88e-4556-aa88-c33f1df0a855 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:41.218347 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293955af-e88e-4556-aa88-c33f1df0a855 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:41.219117 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 775/1559] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:42:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:42:41.219840 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ceecaa-aad5-4e4d-be02-c8b008f32ac7 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:41.220728 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293955af-e88e-4556-aa88-c33f1df0a855 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:41.233993 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ceecaa-aad5-4e4d-be02-c8b008f32ac7 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:41.238678 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb00d97-3099-4324-8886-9f8cc3dfd6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:41.239500 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb00d97-3099-4324-8886-9f8cc3dfd6e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:41.239864 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb00d97-3099-4324-8886-9f8cc3dfd6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:41.240313 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb00d97-3099-4324-8886-9f8cc3dfd6e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:41.241181 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb00d97-3099-4324-8886-9f8cc3dfd6e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:42:41.241529 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb00d97-3099-4324-8886-9f8cc3dfd6e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:41.244131 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb00d97-3099-4324-8886-9f8cc3dfd6e6 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:41.257472 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 785/1560] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:42:41 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:42:41.257472 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee9bea-0503-42f9-a90f-e8599a18c242 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:41.257472 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee9bea-0503-42f9-a90f-e8599a18c242 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:41.257472 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee9bea-0503-42f9-a90f-e8599a18c242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:41.260609 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 776/1561] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:42:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:42:41.261834 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c5cda1-161d-400f-ab4d-6d4ae739ead5 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:41.262131 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c5cda1-161d-400f-ab4d-6d4ae739ead5 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:41.262415 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c5cda1-161d-400f-ab4d-6d4ae739ead5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:41.281279 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-cbee9bea-0503-42f9-a90f-e8599a18c242 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:41.282487 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-cbee9bea-0503-42f9-a90f-e8599a18c242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:41.284831 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-47c5cda1-161d-400f-ab4d-6d4ae739ead5 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:41.286242 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-47c5cda1-161d-400f-ab4d-6d4ae739ead5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:41.331080 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:42:41.346173 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 786/1562] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:41.347060 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee9bea-0503-42f9-a90f-e8599a18c242 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:41.352970 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 777/1563] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:41.353914 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c5cda1-161d-400f-ab4d-6d4ae739ead5 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:41.358690 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:42:41.360397 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee9bea-0503-42f9-a90f-e8599a18c242 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:41.364248 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c5cda1-161d-400f-ab4d-6d4ae739ead5 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:41.365827 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0334c7df-9920-4160-82f3-09bc77e19b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:41.366494 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334c7df-9920-4160-82f3-09bc77e19b7d admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:41.366813 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334c7df-9920-4160-82f3-09bc77e19b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:41.367247 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334c7df-9920-4160-82f3-09bc77e19b7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:41.367980 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0334c7df-9920-4160-82f3-09bc77e19b7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:42:41.368292 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0334c7df-9920-4160-82f3-09bc77e19b7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:41.369727 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d506d-1fe2-4a02-9a44-35695353d5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:41.369924 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0334c7df-9920-4160-82f3-09bc77e19b7d admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:41.370373 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d506d-1fe2-4a02-9a44-35695353d5a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:41.370660 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d506d-1fe2-4a02-9a44-35695353d5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:41.370938 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d506d-1fe2-4a02-9a44-35695353d5a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:41.371663 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d506d-1fe2-4a02-9a44-35695353d5a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:41.373542 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d506d-1fe2-4a02-9a44-35695353d5a7 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:41.386203 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 787/1564] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:42:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:42:41.387933 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0334c7df-9920-4160-82f3-09bc77e19b7d admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:41.398982 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0334c7df-9920-4160-82f3-09bc77e19b7d admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:41.403236 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 778/1565] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:42:41 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:42:41.404060 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d506d-1fe2-4a02-9a44-35695353d5a7 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:41.405096 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26c4878-41d1-48e5-9cca-259b3532f097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:41.405883 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c4878-41d1-48e5-9cca-259b3532f097 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:41.406267 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c4878-41d1-48e5-9cca-259b3532f097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:41.406552 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c4878-41d1-48e5-9cca-259b3532f097 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:41.407334 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26c4878-41d1-48e5-9cca-259b3532f097 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:41.409008 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26c4878-41d1-48e5-9cca-259b3532f097 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:41.424790 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 788/1566] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:42:41.424790 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26c4878-41d1-48e5-9cca-259b3532f097 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:41.432476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26c4878-41d1-48e5-9cca-259b3532f097 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:41.479556 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d506d-1fe2-4a02-9a44-35695353d5a7 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:41.484211 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48d712-82e4-4689-8e12-f8f83921cec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:41.484871 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48d712-82e4-4689-8e12-f8f83921cec2 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:41.485156 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48d712-82e4-4689-8e12-f8f83921cec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:41.485464 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48d712-82e4-4689-8e12-f8f83921cec2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:41.486155 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c48d712-82e4-4689-8e12-f8f83921cec2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:41.488492 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c48d712-82e4-4689-8e12-f8f83921cec2 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:41.491398 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0529374c-0648-46fe-b5d6-f07885017cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:41.492078 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529374c-0648-46fe-b5d6-f07885017cbf admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:41.492367 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529374c-0648-46fe-b5d6-f07885017cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:41.492640 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529374c-0648-46fe-b5d6-f07885017cbf admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:41.493336 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0529374c-0648-46fe-b5d6-f07885017cbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:41.495314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0529374c-0648-46fe-b5d6-f07885017cbf admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:41.517005 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 779/1567] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:42:41 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:42:41.520548 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 789/1568] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:42:41 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:42:41.520688 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48d712-82e4-4689-8e12-f8f83921cec2 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:41.520877 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0529374c-0648-46fe-b5d6-f07885017cbf admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:41.529138 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0529374c-0648-46fe-b5d6-f07885017cbf admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:41.587820 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48d712-82e4-4689-8e12-f8f83921cec2 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:41.588907 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e15ee-8604-4d08-93ce-63383ad4f96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:41.589587 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e15ee-8604-4d08-93ce-63383ad4f96b admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:41.589906 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e15ee-8604-4d08-93ce-63383ad4f96b admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:41.590194 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e15ee-8604-4d08-93ce-63383ad4f96b admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:41.591037 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31e15ee-8604-4d08-93ce-63383ad4f96b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:42:41.591482 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31e15ee-8604-4d08-93ce-63383ad4f96b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:41.592532 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b28b23c-76ee-4c09-ab87-a89ff0ff725e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:41.593181 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31e15ee-8604-4d08-93ce-63383ad4f96b admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:41.593288 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28b23c-76ee-4c09-ab87-a89ff0ff725e admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:41.593460 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28b23c-76ee-4c09-ab87-a89ff0ff725e admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:41.593762 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28b23c-76ee-4c09-ab87-a89ff0ff725e admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:41.594454 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b28b23c-76ee-4c09-ab87-a89ff0ff725e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:41.596990 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b28b23c-76ee-4c09-ab87-a89ff0ff725e admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:41.605108 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 790/1569] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:42:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:42:41.605814 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e15ee-8604-4d08-93ce-63383ad4f96b admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:41.616720 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e15ee-8604-4d08-93ce-63383ad4f96b admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:41.621910 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa10b4-0079-4bd7-ae03-ba836560d641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:41.622566 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa10b4-0079-4bd7-ae03-ba836560d641 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:41.622874 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa10b4-0079-4bd7-ae03-ba836560d641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:41.623235 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa10b4-0079-4bd7-ae03-ba836560d641 admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:41.623920 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aa10b4-0079-4bd7-ae03-ba836560d641 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:42:41.624260 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aa10b4-0079-4bd7-ae03-ba836560d641 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:41.625860 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aa10b4-0079-4bd7-ae03-ba836560d641 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:41.633113 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 780/1570] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:42:41 2022] POST /identity/v3/projects => generated 371 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:42:41.634755 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b28b23c-76ee-4c09-ab87-a89ff0ff725e admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:41.639089 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 791/1571] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:42:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:42:41.646079 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa10b4-0079-4bd7-ae03-ba836560d641 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:41.650608 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa10b4-0079-4bd7-ae03-ba836560d641 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:41.693933 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:42:41.696282 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:42:41.710795 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b28b23c-76ee-4c09-ab87-a89ff0ff725e admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:41.714280 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9fde6b-b2f2-4c22-a72c-4192d9dc78e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:41.715016 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9fde6b-b2f2-4c22-a72c-4192d9dc78e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:41.715322 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9fde6b-b2f2-4c22-a72c-4192d9dc78e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:41.715587 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9fde6b-b2f2-4c22-a72c-4192d9dc78e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:41.716271 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9fde6b-b2f2-4c22-a72c-4192d9dc78e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:41.716431 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4640c933-6297-4eb9-866b-5da8552096f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:41.717011 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640c933-6297-4eb9-866b-5da8552096f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:41.717294 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640c933-6297-4eb9-866b-5da8552096f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:41.717571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640c933-6297-4eb9-866b-5da8552096f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:41.718212 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9fde6b-b2f2-4c22-a72c-4192d9dc78e3 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:41.718352 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4640c933-6297-4eb9-866b-5da8552096f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:41.720158 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4640c933-6297-4eb9-866b-5da8552096f2 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:41.741051 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 792/1572] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:42:41 2022] POST /identity/v3/users => generated 283 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:42:41.744496 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9fde6b-b2f2-4c22-a72c-4192d9dc78e3 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:41.752374 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9fde6b-b2f2-4c22-a72c-4192d9dc78e3 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:41.755642 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 781/1573] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:42:41 2022] POST /identity/v3/projects => generated 387 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:42:41.757181 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4640c933-6297-4eb9-866b-5da8552096f2 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:41.758182 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c54f15-883e-40e9-a750-6308db3729e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:41.758849 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c54f15-883e-40e9-a750-6308db3729e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:41.759169 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c54f15-883e-40e9-a750-6308db3729e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:41.759566 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c54f15-883e-40e9-a750-6308db3729e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:41.760224 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c54f15-883e-40e9-a750-6308db3729e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:41.761827 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c54f15-883e-40e9-a750-6308db3729e6 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:41.774317 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 793/1574] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:42:41.774317 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c54f15-883e-40e9-a750-6308db3729e6 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:41.788320 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c54f15-883e-40e9-a750-6308db3729e6 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:41.822092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4640c933-6297-4eb9-866b-5da8552096f2 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:41.826862 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a23581-3b64-4bf1-92b0-5729ce8653a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:41.827575 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a23581-3b64-4bf1-92b0-5729ce8653a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:41.827868 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a23581-3b64-4bf1-92b0-5729ce8653a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:41.828152 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a23581-3b64-4bf1-92b0-5729ce8653a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:41.828845 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a23581-3b64-4bf1-92b0-5729ce8653a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:41.829873 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: WARNING py.warnings [None req-48a23581-3b64-4bf1-92b0-5729ce8653a6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:42:41.829873 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: warnings.warn(msg) Jun 15 14:42:41.829873 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: Jun 15 14:42:41.831108 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a23581-3b64-4bf1-92b0-5729ce8653a6 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:41.850763 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02ea43-4a50-4547-95c2-a768b4d9a4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:41.851205 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 782/1575] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:42:41 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:42:41.851629 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02ea43-4a50-4547-95c2-a768b4d9a4f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:41.851911 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02ea43-4a50-4547-95c2-a768b4d9a4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:41.852013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a23581-3b64-4bf1-92b0-5729ce8653a6 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:41.852394 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02ea43-4a50-4547-95c2-a768b4d9a4f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:41.853157 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb02ea43-4a50-4547-95c2-a768b4d9a4f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:41.855273 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb02ea43-4a50-4547-95c2-a768b4d9a4f5 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:41.862958 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a23581-3b64-4bf1-92b0-5729ce8653a6 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:41.866712 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 794/1576] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:42:41.868637 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a1b9c-e0c0-4c57-a928-6ba93e0f1017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:41.869295 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a1b9c-e0c0-4c57-a928-6ba93e0f1017 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:41.869581 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a1b9c-e0c0-4c57-a928-6ba93e0f1017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:41.869877 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a1b9c-e0c0-4c57-a928-6ba93e0f1017 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:41.870352 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02ea43-4a50-4547-95c2-a768b4d9a4f5 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:41.870544 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a1b9c-e0c0-4c57-a928-6ba93e0f1017 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:41.872475 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a1b9c-e0c0-4c57-a928-6ba93e0f1017 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:41.879903 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02ea43-4a50-4547-95c2-a768b4d9a4f5 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:41.884503 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378cb32-34fd-456a-9d91-54abddf5e47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:41.885159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378cb32-34fd-456a-9d91-54abddf5e47e admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:41.885443 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378cb32-34fd-456a-9d91-54abddf5e47e admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:41.885717 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378cb32-34fd-456a-9d91-54abddf5e47e admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:41.886404 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378cb32-34fd-456a-9d91-54abddf5e47e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:41.888349 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378cb32-34fd-456a-9d91-54abddf5e47e admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:41.907080 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 783/1577] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:42:41 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:42:41.908040 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a1b9c-e0c0-4c57-a928-6ba93e0f1017 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:41.910285 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 795/1578] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:42:41 2022] POST /identity/v3/users => generated 286 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:42:41.911193 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378cb32-34fd-456a-9d91-54abddf5e47e admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:41.921449 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378cb32-34fd-456a-9d91-54abddf5e47e admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:41.975718 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a1b9c-e0c0-4c57-a928-6ba93e0f1017 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:41.980785 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07593759-89e5-4a2d-a0cd-da0ea6b0c23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:41.981451 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07593759-89e5-4a2d-a0cd-da0ea6b0c23b admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:41.981737 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07593759-89e5-4a2d-a0cd-da0ea6b0c23b admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:41.982020 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07593759-89e5-4a2d-a0cd-da0ea6b0c23b admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:41.982713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07593759-89e5-4a2d-a0cd-da0ea6b0c23b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:41.983843 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd59d13-4dfe-485f-a8d8-d62559a55fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:41.984645 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd59d13-4dfe-485f-a8d8-d62559a55fba admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:41.984897 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07593759-89e5-4a2d-a0cd-da0ea6b0c23b admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:41.985024 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd59d13-4dfe-485f-a8d8-d62559a55fba admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:41.985316 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd59d13-4dfe-485f-a8d8-d62559a55fba admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:41.986151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd59d13-4dfe-485f-a8d8-d62559a55fba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:41.987959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd59d13-4dfe-485f-a8d8-d62559a55fba admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:41.998208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 796/1579] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:42:41.999028 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd59d13-4dfe-485f-a8d8-d62559a55fba admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:42.011110 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd59d13-4dfe-485f-a8d8-d62559a55fba admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:42.015499 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 784/1580] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:42:41 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:42:42.016456 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07593759-89e5-4a2d-a0cd-da0ea6b0c23b admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:42.019043 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb0954f-e937-4e30-8f91-23e4af3fad75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:42.019725 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0954f-e937-4e30-8f91-23e4af3fad75 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:42.020012 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0954f-e937-4e30-8f91-23e4af3fad75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:42.020295 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0954f-e937-4e30-8f91-23e4af3fad75 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:42.020997 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb0954f-e937-4e30-8f91-23e4af3fad75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:42.023525 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb0954f-e937-4e30-8f91-23e4af3fad75 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:42.049857 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 797/1581] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:42:41 2022] POST /identity/v3/users => generated 294 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:42:42.049857 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb0954f-e937-4e30-8f91-23e4af3fad75 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:42.050202 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:42:42.060798 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb0954f-e937-4e30-8f91-23e4af3fad75 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:42.073524 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:42:42.085695 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07593759-89e5-4a2d-a0cd-da0ea6b0c23b admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:42.090693 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c005b0-d714-4c4e-81ae-ba65b1f240a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:42.091632 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c005b0-d714-4c4e-81ae-ba65b1f240a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:42.091912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c005b0-d714-4c4e-81ae-ba65b1f240a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:42.092218 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c005b0-d714-4c4e-81ae-ba65b1f240a6 admin admin] PATH_INFO: `/v3/projects/a243ba2c3f774c10b77a052075497b92/users/c9a3a2e49fee4ea08ca790006d0b29e4/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:42.111757 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c005b0-d714-4c4e-81ae-ba65b1f240a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a243ba2c3f774c10b77a052075497b92, user_id=c9a3a2e49fee4ea08ca790006d0b29e4, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:42.117657 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c005b0-d714-4c4e-81ae-ba65b1f240a6 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:42.123870 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f29140-38b8-4e17-a05d-a6abece643d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:42.124634 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f29140-38b8-4e17-a05d-a6abece643d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:42.124928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f29140-38b8-4e17-a05d-a6abece643d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:42.125279 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f29140-38b8-4e17-a05d-a6abece643d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:42.126090 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f29140-38b8-4e17-a05d-a6abece643d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:42.129627 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f29140-38b8-4e17-a05d-a6abece643d3 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:42.138565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 785/1582] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:42:42 2022] PUT /identity/v3/projects/a243ba2c3f774c10b77a052075497b92/users/c9a3a2e49fee4ea08ca790006d0b29e4/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:42:42.143140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c005b0-d714-4c4e-81ae-ba65b1f240a6 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:42.148468 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 798/1583] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:42:42.148468 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f29140-38b8-4e17-a05d-a6abece643d3 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:42.158369 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f29140-38b8-4e17-a05d-a6abece643d3 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:42.211260 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c005b0-d714-4c4e-81ae-ba65b1f240a6 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:42.217909 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1e958-10c1-48ed-ab0b-ec57f923f7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:42.218854 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1e958-10c1-48ed-ab0b-ec57f923f7f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:42.219298 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83345330-1edc-4d78-8c3a-db106292355f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:42.219431 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1e958-10c1-48ed-ab0b-ec57f923f7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:42.219844 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83345330-1edc-4d78-8c3a-db106292355f admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:42.219941 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1e958-10c1-48ed-ab0b-ec57f923f7f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:42.220379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83345330-1edc-4d78-8c3a-db106292355f admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:42.220379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83345330-1edc-4d78-8c3a-db106292355f admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:42.221123 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83345330-1edc-4d78-8c3a-db106292355f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:42.221505 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e1e958-10c1-48ed-ab0b-ec57f923f7f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:42.223262 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83345330-1edc-4d78-8c3a-db106292355f admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:42.223481 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e1e958-10c1-48ed-ab0b-ec57f923f7f0 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:42.235105 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 786/1584] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:42:42.235849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1e958-10c1-48ed-ab0b-ec57f923f7f0 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:42.246168 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1e958-10c1-48ed-ab0b-ec57f923f7f0 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:42.246372 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 799/1585] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:42:42 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:42:42.247366 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83345330-1edc-4d78-8c3a-db106292355f admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:42.251354 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e4a96-e7fd-4308-aa6c-df93a7bdef3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:42.252017 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e4a96-e7fd-4308-aa6c-df93a7bdef3c admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:42.252306 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e4a96-e7fd-4308-aa6c-df93a7bdef3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:42.252587 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e4a96-e7fd-4308-aa6c-df93a7bdef3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:42.253374 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7e4a96-e7fd-4308-aa6c-df93a7bdef3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:42.255104 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7e4a96-e7fd-4308-aa6c-df93a7bdef3c admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:42.257938 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83345330-1edc-4d78-8c3a-db106292355f admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:42.263339 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074c0ce-4173-400e-ad19-ac039d7454b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:42.264041 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074c0ce-4173-400e-ad19-ac039d7454b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:42.264325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074c0ce-4173-400e-ad19-ac039d7454b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:42.264603 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074c0ce-4173-400e-ad19-ac039d7454b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:42.265331 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6074c0ce-4173-400e-ad19-ac039d7454b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:42.266842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 787/1586] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:42:42.266981 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6074c0ce-4173-400e-ad19-ac039d7454b3 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:42.267702 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e4a96-e7fd-4308-aa6c-df93a7bdef3c admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:42.278538 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e4a96-e7fd-4308-aa6c-df93a7bdef3c admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:42.278710 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 800/1587] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:42:42.279621 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074c0ce-4173-400e-ad19-ac039d7454b3 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:42.283417 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aff6f0-ead9-40f0-972f-97e3bfc2c395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:42.284080 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aff6f0-ead9-40f0-972f-97e3bfc2c395 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:42.284368 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aff6f0-ead9-40f0-972f-97e3bfc2c395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:42.284650 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aff6f0-ead9-40f0-972f-97e3bfc2c395 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:42.285476 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aff6f0-ead9-40f0-972f-97e3bfc2c395 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:42.288253 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aff6f0-ead9-40f0-972f-97e3bfc2c395 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:42.294476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074c0ce-4173-400e-ad19-ac039d7454b3 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:42.301125 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635a0f4-678c-4631-aa71-31a114f6363a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:42.301804 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635a0f4-678c-4631-aa71-31a114f6363a admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:42.302094 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635a0f4-678c-4631-aa71-31a114f6363a admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:42.302373 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635a0f4-678c-4631-aa71-31a114f6363a admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:42.303147 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1635a0f4-678c-4631-aa71-31a114f6363a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:42.304945 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1635a0f4-678c-4631-aa71-31a114f6363a admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:42.317784 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 801/1588] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:42:42.319233 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2aa90-5842-48fa-ad45-ccd04a2f5950 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:42.319524 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2aa90-5842-48fa-ad45-ccd04a2f5950 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:42.319801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2aa90-5842-48fa-ad45-ccd04a2f5950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:42.323563 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 788/1589] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:42:42 2022] POST /identity/v3/users => generated 287 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:42:42.323563 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aff6f0-ead9-40f0-972f-97e3bfc2c395 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:42.330939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aff6f0-ead9-40f0-972f-97e3bfc2c395 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:42.336455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b14f996-745f-4cdf-bc0b-37a6df2fbe35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:42.337140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14f996-745f-4cdf-bc0b-37a6df2fbe35 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:42.337281 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-63e2aa90-5842-48fa-ad45-ccd04a2f5950 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:42.337424 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14f996-745f-4cdf-bc0b-37a6df2fbe35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:42.337703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14f996-745f-4cdf-bc0b-37a6df2fbe35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:42.338408 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-63e2aa90-5842-48fa-ad45-ccd04a2f5950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:42.338520 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b14f996-745f-4cdf-bc0b-37a6df2fbe35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:42.340075 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b14f996-745f-4cdf-bc0b-37a6df2fbe35 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:42.349320 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 802/1590] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:42.353799 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2aa90-5842-48fa-ad45-ccd04a2f5950 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:42.353894 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 789/1591] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:42:42.353894 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b14f996-745f-4cdf-bc0b-37a6df2fbe35 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:42.363425 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2aa90-5842-48fa-ad45-ccd04a2f5950 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:42.368154 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81883bda-56a8-477c-9428-d1c35507de96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:42.369049 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81883bda-56a8-477c-9428-d1c35507de96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:42.369328 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81883bda-56a8-477c-9428-d1c35507de96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:42.369596 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81883bda-56a8-477c-9428-d1c35507de96 admin admin] PATH_INFO: `/v3/projects/f35be82fde6249bab9008a41cd7ce8d9/users/e452beb1eaf540daa9bbfec92e3138cb/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:42.370664 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b14f996-745f-4cdf-bc0b-37a6df2fbe35 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:42.375325 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acdd39c-083d-404e-893e-34fc1ccdc660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:42.376019 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdd39c-083d-404e-893e-34fc1ccdc660 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:42.376306 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdd39c-083d-404e-893e-34fc1ccdc660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:42.376580 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdd39c-083d-404e-893e-34fc1ccdc660 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:42.377308 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acdd39c-083d-404e-893e-34fc1ccdc660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:42.378903 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acdd39c-083d-404e-893e-34fc1ccdc660 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:42.384160 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81883bda-56a8-477c-9428-d1c35507de96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f35be82fde6249bab9008a41cd7ce8d9, user_id=e452beb1eaf540daa9bbfec92e3138cb, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:42.385205 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: WARNING py.warnings [None req-81883bda-56a8-477c-9428-d1c35507de96 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:42:42.385205 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: warnings.warn(msg) Jun 15 14:42:42.385205 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: Jun 15 14:42:42.387200 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81883bda-56a8-477c-9428-d1c35507de96 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:42.389581 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 790/1592] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:42:42.390398 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acdd39c-083d-404e-893e-34fc1ccdc660 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:42.402949 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 803/1593] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:42:42 2022] PUT /identity/v3/projects/f35be82fde6249bab9008a41cd7ce8d9/users/e452beb1eaf540daa9bbfec92e3138cb/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:42:42.403203 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acdd39c-083d-404e-893e-34fc1ccdc660 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:42.404643 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81883bda-56a8-477c-9428-d1c35507de96 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:42.454394 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:42:42.457524 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:42:42.465501 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088f332e-1446-4e96-827a-a3fe176337ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:42.466470 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f332e-1446-4e96-827a-a3fe176337ef admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:42.466847 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f332e-1446-4e96-827a-a3fe176337ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:42.467517 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f332e-1446-4e96-827a-a3fe176337ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:42.468442 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088f332e-1446-4e96-827a-a3fe176337ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:42.470010 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81883bda-56a8-477c-9428-d1c35507de96 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:42.470272 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088f332e-1446-4e96-827a-a3fe176337ef admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:42.475463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7f6e0-4f6c-4b24-82c4-7a5808068324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:42.476496 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7f6e0-4f6c-4b24-82c4-7a5808068324 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:42.476649 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7f6e0-4f6c-4b24-82c4-7a5808068324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:42.476926 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7f6e0-4f6c-4b24-82c4-7a5808068324 admin admin] PATH_INFO: `/v3/projects/4d36485f72ad45ccbe7bf55f13fbf59d/users/98c07f9a7da44055959709fd84f3042e/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:42.481146 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 791/1594] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:42:42.482000 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088f332e-1446-4e96-827a-a3fe176337ef admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:42.491638 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088f332e-1446-4e96-827a-a3fe176337ef admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:42.493767 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd7f6e0-4f6c-4b24-82c4-7a5808068324 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d36485f72ad45ccbe7bf55f13fbf59d, user_id=98c07f9a7da44055959709fd84f3042e, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:42.496416 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea5a74-e547-4344-80e1-39905827713e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:42.496662 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd7f6e0-4f6c-4b24-82c4-7a5808068324 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:42.497071 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea5a74-e547-4344-80e1-39905827713e admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:42.497353 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea5a74-e547-4344-80e1-39905827713e admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:42.497632 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea5a74-e547-4344-80e1-39905827713e admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:42.498357 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ea5a74-e547-4344-80e1-39905827713e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:42.499979 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ea5a74-e547-4344-80e1-39905827713e admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:42.514165 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 804/1595] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:42:42 2022] PUT /identity/v3/projects/4d36485f72ad45ccbe7bf55f13fbf59d/users/98c07f9a7da44055959709fd84f3042e/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:42:42.516196 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7f6e0-4f6c-4b24-82c4-7a5808068324 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:42.521019 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 792/1596] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:42:42.521019 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea5a74-e547-4344-80e1-39905827713e admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:42.530092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea5a74-e547-4344-80e1-39905827713e admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:42.587863 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7f6e0-4f6c-4b24-82c4-7a5808068324 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:42.591962 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781221f0-d86a-4387-8b32-0c59d272f36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:42.592746 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368cd44e-8b12-4cf4-8b98-160b8b854ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:42.592861 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781221f0-d86a-4387-8b32-0c59d272f36e admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:42.593129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781221f0-d86a-4387-8b32-0c59d272f36e admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:42.593409 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368cd44e-8b12-4cf4-8b98-160b8b854ec7 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:42.593522 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781221f0-d86a-4387-8b32-0c59d272f36e admin admin] PATH_INFO: `/v3/projects/c1f8a1b95df743588b298fe0efbcd967/users/627ebb2f052145cb8a009e1009d15da7/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:42.593692 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368cd44e-8b12-4cf4-8b98-160b8b854ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:42.593966 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368cd44e-8b12-4cf4-8b98-160b8b854ec7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:42.594688 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368cd44e-8b12-4cf4-8b98-160b8b854ec7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:42.597079 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368cd44e-8b12-4cf4-8b98-160b8b854ec7 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:42.607284 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 805/1597] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:42:42.607994 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368cd44e-8b12-4cf4-8b98-160b8b854ec7 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:42.611420 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-368cd44e-8b12-4cf4-8b98-160b8b854ec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:42.611594 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781221f0-d86a-4387-8b32-0c59d272f36e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1f8a1b95df743588b298fe0efbcd967, user_id=627ebb2f052145cb8a009e1009d15da7, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:42.614268 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781221f0-d86a-4387-8b32-0c59d272f36e admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:42.628966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 793/1598] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:42:42 2022] PUT /identity/v3/projects/c1f8a1b95df743588b298fe0efbcd967/users/627ebb2f052145cb8a009e1009d15da7/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:42:42.635583 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781221f0-d86a-4387-8b32-0c59d272f36e admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:42.689725 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368cd44e-8b12-4cf4-8b98-160b8b854ec7 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:42.698896 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781221f0-d86a-4387-8b32-0c59d272f36e admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:42.707701 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86dda4-ce74-4931-977d-85cf789b9d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:42.708476 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86dda4-ce74-4931-977d-85cf789b9d23 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:42.708769 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86dda4-ce74-4931-977d-85cf789b9d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:42.709649 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86dda4-ce74-4931-977d-85cf789b9d23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:42.710398 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc86dda4-ce74-4931-977d-85cf789b9d23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:42.712067 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc86dda4-ce74-4931-977d-85cf789b9d23 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:42.727341 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 794/1599] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:42:42.729417 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a1340-0f6c-4320-ba8b-c73e1afb2601 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:42.729417 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a1340-0f6c-4320-ba8b-c73e1afb2601 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:42.729575 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a1340-0f6c-4320-ba8b-c73e1afb2601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:42.749021 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-a94a1340-0f6c-4320-ba8b-c73e1afb2601 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:42.750242 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-a94a1340-0f6c-4320-ba8b-c73e1afb2601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:42.753724 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c26f11e-4c85-4e8e-9b57-2e67a5882e99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:42.754385 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26f11e-4c85-4e8e-9b57-2e67a5882e99 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:42.754656 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26f11e-4c85-4e8e-9b57-2e67a5882e99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:42.754930 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26f11e-4c85-4e8e-9b57-2e67a5882e99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:42.758045 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 795/1600] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:42.759827 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94a1340-0f6c-4320-ba8b-c73e1afb2601 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:42.766973 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c26f11e-4c85-4e8e-9b57-2e67a5882e99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:42.768546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c26f11e-4c85-4e8e-9b57-2e67a5882e99 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:42.769754 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94a1340-0f6c-4320-ba8b-c73e1afb2601 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:42.774700 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf361e16-1728-4729-909e-2737a4894b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:42.775906 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf361e16-1728-4729-909e-2737a4894b28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:42.776216 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf361e16-1728-4729-909e-2737a4894b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:42.776489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf361e16-1728-4729-909e-2737a4894b28 admin admin] PATH_INFO: `/v3/projects/71b718b10be64311a6fc49c19b937c58/users/db037e07d96746b78c3e67376649a657/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:42.778799 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 806/1601] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:42:42.779648 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c26f11e-4c85-4e8e-9b57-2e67a5882e99 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:42.789999 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c26f11e-4c85-4e8e-9b57-2e67a5882e99 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:42.795437 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfae3fb-a832-430c-8350-b60e046ed948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:42.796524 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfae3fb-a832-430c-8350-b60e046ed948 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:42.797151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfae3fb-a832-430c-8350-b60e046ed948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:42.797429 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfae3fb-a832-430c-8350-b60e046ed948 admin admin] PATH_INFO: `/v3/projects/54d57ba7dd0d441b9748fc600bc34972/users/25ba370b8daf49e48514cd49c2851215/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:42.799705 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf361e16-1728-4729-909e-2737a4894b28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71b718b10be64311a6fc49c19b937c58, user_id=db037e07d96746b78c3e67376649a657, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:42.802478 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf361e16-1728-4729-909e-2737a4894b28 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:42.820187 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfae3fb-a832-430c-8350-b60e046ed948 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54d57ba7dd0d441b9748fc600bc34972, user_id=25ba370b8daf49e48514cd49c2851215, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:42.820800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 796/1602] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:42:42 2022] PUT /identity/v3/projects/71b718b10be64311a6fc49c19b937c58/users/db037e07d96746b78c3e67376649a657/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:42:42.822144 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a63da2-6682-4d9d-8bcc-99793a10cb56 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:42.822622 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a63da2-6682-4d9d-8bcc-99793a10cb56 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:42.822893 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a63da2-6682-4d9d-8bcc-99793a10cb56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:42.824086 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfae3fb-a832-430c-8350-b60e046ed948 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:42.846118 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 807/1603] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:42:42 2022] PUT /identity/v3/projects/54d57ba7dd0d441b9748fc600bc34972/users/25ba370b8daf49e48514cd49c2851215/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:42:42.846118 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfae3fb-a832-430c-8350-b60e046ed948 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:42.852105 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:42:42.858242 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-91a63da2-6682-4d9d-8bcc-99793a10cb56 None None] MFA Rules not processed for user `98c07f9a7da44055959709fd84f3042e`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:42.859574 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-91a63da2-6682-4d9d-8bcc-99793a10cb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:42.879539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:42:42.931778 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfae3fb-a832-430c-8350-b60e046ed948 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:42.937057 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 797/1604] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:42 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:42.938461 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d256dc9-3d24-466c-8dba-6868a501935f None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:42.938819 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d256dc9-3d24-466c-8dba-6868a501935f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:42.939216 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d256dc9-3d24-466c-8dba-6868a501935f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:42.941639 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f8fa5-01ff-4b95-9e1f-2a33e80494c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:42.942691 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f8fa5-01ff-4b95-9e1f-2a33e80494c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:42.944098 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f8fa5-01ff-4b95-9e1f-2a33e80494c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:42.944304 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f8fa5-01ff-4b95-9e1f-2a33e80494c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:42.945035 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6f8fa5-01ff-4b95-9e1f-2a33e80494c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:42.947218 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6f8fa5-01ff-4b95-9e1f-2a33e80494c8 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:42.965227 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 808/1605] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:42:42.969534 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f8fa5-01ff-4b95-9e1f-2a33e80494c8 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:42.974461 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-6d256dc9-3d24-466c-8dba-6868a501935f None None] MFA Rules not processed for user `627ebb2f052145cb8a009e1009d15da7`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:42.976474 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-6d256dc9-3d24-466c-8dba-6868a501935f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:42.981772 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f8fa5-01ff-4b95-9e1f-2a33e80494c8 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:42.992623 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4829dd81-0a02-4c51-b443-fcff2a6cdd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:42.993611 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4829dd81-0a02-4c51-b443-fcff2a6cdd3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:42.994057 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4829dd81-0a02-4c51-b443-fcff2a6cdd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:42.994301 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4829dd81-0a02-4c51-b443-fcff2a6cdd3b admin admin] PATH_INFO: `/v3/projects/6d3ce7cecd2f4983b8442f70ed2afdb1/users/1efb0951ba1144d58698e02e0e5e9416/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:43.012773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4829dd81-0a02-4c51-b443-fcff2a6cdd3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d3ce7cecd2f4983b8442f70ed2afdb1, user_id=1efb0951ba1144d58698e02e0e5e9416, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:43.016991 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4829dd81-0a02-4c51-b443-fcff2a6cdd3b admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:43.047283 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 809/1606] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:42:42 2022] PUT /identity/v3/projects/6d3ce7cecd2f4983b8442f70ed2afdb1/users/1efb0951ba1144d58698e02e0e5e9416/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:42:43.047283 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4829dd81-0a02-4c51-b443-fcff2a6cdd3b admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:43.094538 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 798/1607] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:43.096458 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9db7e-ba91-463a-b6b8-c48c8bcc68da None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:43.096715 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9db7e-ba91-463a-b6b8-c48c8bcc68da None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:43.097075 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9db7e-ba91-463a-b6b8-c48c8bcc68da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:43.132508 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-23d9db7e-ba91-463a-b6b8-c48c8bcc68da None None] MFA Rules not processed for user `db037e07d96746b78c3e67376649a657`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:43.133762 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-23d9db7e-ba91-463a-b6b8-c48c8bcc68da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:43.142912 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4829dd81-0a02-4c51-b443-fcff2a6cdd3b admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:43.153782 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7c02a-7e7a-48f0-80f1-e528cddf77dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:43.154546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7c02a-7e7a-48f0-80f1-e528cddf77dc service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:43.154868 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7c02a-7e7a-48f0-80f1-e528cddf77dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:43.155204 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7c02a-7e7a-48f0-80f1-e528cddf77dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:43.232478 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 799/1608] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:43.238250 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd8a5c-2cc2-485c-8861-4b3c3df0c3a3 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:43.238553 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd8a5c-2cc2-485c-8861-4b3c3df0c3a3 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:43.238874 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd8a5c-2cc2-485c-8861-4b3c3df0c3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:43.248049 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d7c02a-7e7a-48f0-80f1-e528cddf77dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:43.254654 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d7c02a-7e7a-48f0-80f1-e528cddf77dc service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:43.262099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-1efd8a5c-2cc2-485c-8861-4b3c3df0c3a3 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:43.263404 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1efd8a5c-2cc2-485c-8861-4b3c3df0c3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:43.276372 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 810/1609] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:42:43.277467 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7c02a-7e7a-48f0-80f1-e528cddf77dc service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:43.289239 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 800/1610] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:43.290408 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efd8a5c-2cc2-485c-8861-4b3c3df0c3a3 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:43.314345 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efd8a5c-2cc2-485c-8861-4b3c3df0c3a3 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:43.317381 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:42:43.333884 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c72e52-7bdc-40cc-97a6-fd8ee9b0383b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:43.335948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c72e52-7bdc-40cc-97a6-fd8ee9b0383b admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:43.336293 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c72e52-7bdc-40cc-97a6-fd8ee9b0383b admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:43.336647 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c72e52-7bdc-40cc-97a6-fd8ee9b0383b admin admin] PATH_INFO: `/v3/projects/da762c4e79d14510afd0a35251c4bad3/users/9202dcd0010f4727b9a4966a5793db78/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:43.367605 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7c02a-7e7a-48f0-80f1-e528cddf77dc service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:43.372692 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c72e52-7bdc-40cc-97a6-fd8ee9b0383b admin admin] RBAC: Authorizing `identity:create_grant(project_id=da762c4e79d14510afd0a35251c4bad3, user_id=9202dcd0010f4727b9a4966a5793db78, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:43.374390 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe19a0-ddc9-4f0a-a2e5-f0e1056f219a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:43.375224 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe19a0-ddc9-4f0a-a2e5-f0e1056f219a service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:43.375468 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c72e52-7bdc-40cc-97a6-fd8ee9b0383b admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:43.375660 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe19a0-ddc9-4f0a-a2e5-f0e1056f219a service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:43.375812 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe19a0-ddc9-4f0a-a2e5-f0e1056f219a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:43.413589 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 801/1611] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:42:43 2022] PUT /identity/v3/projects/da762c4e79d14510afd0a35251c4bad3/users/9202dcd0010f4727b9a4966a5793db78/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:42:43.415071 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb218f2e-e4d8-4855-8b67-56e3df1e7150 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:43.415371 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb218f2e-e4d8-4855-8b67-56e3df1e7150 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:43.415656 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb218f2e-e4d8-4855-8b67-56e3df1e7150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:43.441072 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-bb218f2e-e4d8-4855-8b67-56e3df1e7150 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:43.442694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-bb218f2e-e4d8-4855-8b67-56e3df1e7150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:43.449743 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:42:43.482072 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe19a0-ddc9-4f0a-a2e5-f0e1056f219a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:43.483602 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe19a0-ddc9-4f0a-a2e5-f0e1056f219a service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:43.497059 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 811/1612] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:43 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:42:43.499784 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75debfc0-7522-45a2-bd3b-c9e5e273e2f1 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:43.500134 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75debfc0-7522-45a2-bd3b-c9e5e273e2f1 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:43.500424 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75debfc0-7522-45a2-bd3b-c9e5e273e2f1 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:43.502380 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 812/1613] 10.222.0.45 () {54 vars in 961 bytes} [Wed Jun 15 14:42:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:42:43.503370 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75debfc0-7522-45a2-bd3b-c9e5e273e2f1 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:43.564506 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 802/1614] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:43.566049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb218f2e-e4d8-4855-8b67-56e3df1e7150 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:43.611967 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75debfc0-7522-45a2-bd3b-c9e5e273e2f1 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:43.617526 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a38536-39aa-48a8-8009-4453ae81c6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:43.618118 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a38536-39aa-48a8-8009-4453ae81c6c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:43.618561 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a38536-39aa-48a8-8009-4453ae81c6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:43.618844 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a38536-39aa-48a8-8009-4453ae81c6c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:43.619937 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a38536-39aa-48a8-8009-4453ae81c6c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:43.621919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a38536-39aa-48a8-8009-4453ae81c6c1 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:43.664759 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 813/1615] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:42:43 2022] POST /identity/v3/projects => generated 371 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:42:43.666712 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ccb4c-a644-4f9d-b3ad-8b169c98ce87 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:43.667241 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ccb4c-a644-4f9d-b3ad-8b169c98ce87 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:43.667666 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ccb4c-a644-4f9d-b3ad-8b169c98ce87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:43.692073 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb218f2e-e4d8-4855-8b67-56e3df1e7150 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:43.709375 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-390ccb4c-a644-4f9d-b3ad-8b169c98ce87 None None] MFA Rules not processed for user `9202dcd0010f4727b9a4966a5793db78`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:43.710952 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-390ccb4c-a644-4f9d-b3ad-8b169c98ce87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:43.826680 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:42:43.851015 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 814/1616] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:43.852381 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b2598-cd8a-47f1-ba2d-6fa02c0b4493 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:43.852669 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b2598-cd8a-47f1-ba2d-6fa02c0b4493 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:43.852951 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b2598-cd8a-47f1-ba2d-6fa02c0b4493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:43.902591 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d80742-baec-4de9-80d0-5f39ba83740a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:43.903763 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d80742-baec-4de9-80d0-5f39ba83740a service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:43.904049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d80742-baec-4de9-80d0-5f39ba83740a service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:43.904327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d80742-baec-4de9-80d0-5f39ba83740a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:43.919528 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-446b2598-cd8a-47f1-ba2d-6fa02c0b4493 None None] MFA Rules not processed for user `f3185bd2458c4e22830975700888e9be`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:43.923232 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-446b2598-cd8a-47f1-ba2d-6fa02c0b4493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:44.073043 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d80742-baec-4de9-80d0-5f39ba83740a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:44.078269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d80742-baec-4de9-80d0-5f39ba83740a service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:44.080252 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 815/1617] 10.222.0.45 () {58 vars in 1058 bytes} [Wed Jun 15 14:42:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:44.081189 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b2598-cd8a-47f1-ba2d-6fa02c0b4493 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:44.093498 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b2598-cd8a-47f1-ba2d-6fa02c0b4493 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:44.098248 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d45850-e5a0-4da0-bc98-9ddd85ae62ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:44.098927 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d45850-e5a0-4da0-bc98-9ddd85ae62ae service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:44.099254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d45850-e5a0-4da0-bc98-9ddd85ae62ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:44.100255 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d45850-e5a0-4da0-bc98-9ddd85ae62ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:44.112734 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 803/1618] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:42:44.113781 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d45850-e5a0-4da0-bc98-9ddd85ae62ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:44.116192 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d80742-baec-4de9-80d0-5f39ba83740a service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:44.116491 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d45850-e5a0-4da0-bc98-9ddd85ae62ae service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:44.133296 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 816/1619] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:42:44.135340 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f338312-6294-4c1e-88d6-5a6f16f12352 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:44.135633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f338312-6294-4c1e-88d6-5a6f16f12352 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:44.135913 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f338312-6294-4c1e-88d6-5a6f16f12352 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:44.137761 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 817/1620] 10.222.0.45 () {54 vars in 961 bytes} [Wed Jun 15 14:42:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:42:44.138274 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f338312-6294-4c1e-88d6-5a6f16f12352 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:44.155978 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d80742-baec-4de9-80d0-5f39ba83740a service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:44.161638 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f338312-6294-4c1e-88d6-5a6f16f12352 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:44.171870 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f54db78-55c1-4937-b438-a2356760803c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:44.172444 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a23d1-5d47-4f1b-8150-f9b7ed0ece5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:44.172814 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54db78-55c1-4937-b438-a2356760803c service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:44.173107 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54db78-55c1-4937-b438-a2356760803c service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:44.173693 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54db78-55c1-4937-b438-a2356760803c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:44.174200 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a23d1-5d47-4f1b-8150-f9b7ed0ece5f admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:44.174781 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a23d1-5d47-4f1b-8150-f9b7ed0ece5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:44.175235 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a23d1-5d47-4f1b-8150-f9b7ed0ece5f admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:44.176160 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a23d1-5d47-4f1b-8150-f9b7ed0ece5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:44.184951 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a23d1-5d47-4f1b-8150-f9b7ed0ece5f admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:44.226216 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:42:44.235244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 804/1621] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:42:44 2022] POST /identity/v3/users => generated 292 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:42:44.236709 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91f2e2-9373-402e-8a8f-c553b86c2874 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:44.238424 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91f2e2-9373-402e-8a8f-c553b86c2874 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:44.238719 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91f2e2-9373-402e-8a8f-c553b86c2874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:44.285895 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f54db78-55c1-4937-b438-a2356760803c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:44.287837 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-5d91f2e2-9373-402e-8a8f-c553b86c2874 None None] MFA Rules not processed for user `f3185bd2458c4e22830975700888e9be`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:44.288685 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f54db78-55c1-4937-b438-a2356760803c service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:44.291090 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-5d91f2e2-9373-402e-8a8f-c553b86c2874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:44.310076 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 818/1622] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:42:44.310076 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f54db78-55c1-4937-b438-a2356760803c service nova] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:44.321904 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f54db78-55c1-4937-b438-a2356760803c service nova] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:44.329186 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa443880-175b-483e-9e5a-f53bf1eb41a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:44.330062 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa443880-175b-483e-9e5a-f53bf1eb41a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:44.330851 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa443880-175b-483e-9e5a-f53bf1eb41a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:44.331343 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa443880-175b-483e-9e5a-f53bf1eb41a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:44.332731 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa443880-175b-483e-9e5a-f53bf1eb41a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:44.335316 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa443880-175b-483e-9e5a-f53bf1eb41a6 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:44.341372 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 805/1623] 10.222.0.45 () {58 vars in 1058 bytes} [Wed Jun 15 14:42:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:44.354542 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 819/1624] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:42:44 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:42:44.354768 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91f2e2-9373-402e-8a8f-c553b86c2874 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:44.356056 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa443880-175b-483e-9e5a-f53bf1eb41a6 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:44.379048 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91f2e2-9373-402e-8a8f-c553b86c2874 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:44.394100 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131d57e-b531-431c-9c30-c6a0da985a48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:44.397014 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131d57e-b531-431c-9c30-c6a0da985a48 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:44.398070 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa443880-175b-483e-9e5a-f53bf1eb41a6 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:44.398480 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131d57e-b531-431c-9c30-c6a0da985a48 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:44.398779 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131d57e-b531-431c-9c30-c6a0da985a48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:44.418742 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da84f7-1626-4c92-af5d-8652dbebbdd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:44.420426 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da84f7-1626-4c92-af5d-8652dbebbdd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:44.420909 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da84f7-1626-4c92-af5d-8652dbebbdd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:44.421508 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da84f7-1626-4c92-af5d-8652dbebbdd8 admin admin] PATH_INFO: `/v3/projects/e2da2cd01ea144699aea0ac22941a9ad/users/021cf3b95f8d4c358b8b61db26d65126/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:44.433073 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f131d57e-b531-431c-9c30-c6a0da985a48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:44.438520 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f131d57e-b531-431c-9c30-c6a0da985a48 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:44.460976 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da84f7-1626-4c92-af5d-8652dbebbdd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2da2cd01ea144699aea0ac22941a9ad, user_id=021cf3b95f8d4c358b8b61db26d65126, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:44.466425 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da84f7-1626-4c92-af5d-8652dbebbdd8 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:44.468441 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 806/1625] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:42:44.518301 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 820/1626] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:42:44 2022] PUT /identity/v3/projects/e2da2cd01ea144699aea0ac22941a9ad/users/021cf3b95f8d4c358b8b61db26d65126/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:42:44.519304 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131d57e-b531-431c-9c30-c6a0da985a48 service nova] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:44.545082 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131d57e-b531-431c-9c30-c6a0da985a48 service nova] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:44.714378 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca1939-7fd7-434c-8e3a-24adbb07f0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:44.715186 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca1939-7fd7-434c-8e3a-24adbb07f0cd admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:44.715486 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca1939-7fd7-434c-8e3a-24adbb07f0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:44.715767 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca1939-7fd7-434c-8e3a-24adbb07f0cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:44.716508 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ca1939-7fd7-434c-8e3a-24adbb07f0cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:44.718122 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ca1939-7fd7-434c-8e3a-24adbb07f0cd admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:44.749234 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 807/1627] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:42:44 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:42:44.750433 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da84f7-1626-4c92-af5d-8652dbebbdd8 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:44.769450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da84f7-1626-4c92-af5d-8652dbebbdd8 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:44.796648 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411027a-9a97-41a1-b47c-7ce587447466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:44.797553 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411027a-9a97-41a1-b47c-7ce587447466 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:44.797836 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411027a-9a97-41a1-b47c-7ce587447466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:44.798110 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411027a-9a97-41a1-b47c-7ce587447466 admin admin] PATH_INFO: `/v3/projects/e2da2cd01ea144699aea0ac22941a9ad/users/021cf3b95f8d4c358b8b61db26d65126/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:44.802276 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411027a-9a97-41a1-b47c-7ce587447466 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2da2cd01ea144699aea0ac22941a9ad, user_id=021cf3b95f8d4c358b8b61db26d65126, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:44.807213 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411027a-9a97-41a1-b47c-7ce587447466 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:44.841682 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 821/1628] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:42:44 2022] PUT /identity/v3/projects/e2da2cd01ea144699aea0ac22941a9ad/users/021cf3b95f8d4c358b8b61db26d65126/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:42:44.857062 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f79db3-6cd4-4c19-a2c2-40bca1ae76ca None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:44.857348 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f79db3-6cd4-4c19-a2c2-40bca1ae76ca None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:44.857711 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f79db3-6cd4-4c19-a2c2-40bca1ae76ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:44.927305 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-41f79db3-6cd4-4c19-a2c2-40bca1ae76ca None None] MFA Rules not processed for user `021cf3b95f8d4c358b8b61db26d65126`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:44.930391 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-41f79db3-6cd4-4c19-a2c2-40bca1ae76ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:45.004093 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea8be3-4e1d-4b43-99af-2e431d893123 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:45.004386 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea8be3-4e1d-4b43-99af-2e431d893123 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:45.004726 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea8be3-4e1d-4b43-99af-2e431d893123 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:45.007550 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 822/1629] 10.222.0.45 () {54 vars in 912 bytes} [Wed Jun 15 14:42:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:42:45.013087 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7237c1e-618c-425d-83d4-2ab70977fa66 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:45.013495 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7237c1e-618c-425d-83d4-2ab70977fa66 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:45.014606 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7237c1e-618c-425d-83d4-2ab70977fa66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:45.058637 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-c7237c1e-618c-425d-83d4-2ab70977fa66 None None] MFA Rules not processed for user `f3185bd2458c4e22830975700888e9be`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:45.059894 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c7237c1e-618c-425d-83d4-2ab70977fa66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:45.064701 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 808/1630] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:44 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:45.066808 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d50d41-2273-4772-901a-28c86df45b43 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:45.067401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d50d41-2273-4772-901a-28c86df45b43 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:45.067826 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d50d41-2273-4772-901a-28c86df45b43 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:45.081930 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 809/1631] 10.222.0.45 () {54 vars in 912 bytes} [Wed Jun 15 14:42:45 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:42:45.083248 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d50d41-2273-4772-901a-28c86df45b43 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:45.122224 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:42:45.191847 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d50d41-2273-4772-901a-28c86df45b43 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:45.197637 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83913676-abf3-461d-b054-7c1860ffcaaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:45.198626 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83913676-abf3-461d-b054-7c1860ffcaaa service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:45.199152 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83913676-abf3-461d-b054-7c1860ffcaaa service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:45.199609 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83913676-abf3-461d-b054-7c1860ffcaaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:45.211875 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83913676-abf3-461d-b054-7c1860ffcaaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:45.214716 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83913676-abf3-461d-b054-7c1860ffcaaa service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:45.244330 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 810/1632] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:45 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:42:45.247460 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9525ae22-a9ac-4e9a-8800-0cb80d03eb36 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:45.250543 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9525ae22-a9ac-4e9a-8800-0cb80d03eb36 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:45.251109 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9525ae22-a9ac-4e9a-8800-0cb80d03eb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:45.264136 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 823/1633] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Jun 15 14:42:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:42:45.275752 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:42:45.279378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7237c1e-618c-425d-83d4-2ab70977fa66 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:45.300233 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7237c1e-618c-425d-83d4-2ab70977fa66 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:45.313976 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52788c-d5fc-4e6f-9fb5-70481a48dae8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:45.314436 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-9525ae22-a9ac-4e9a-8800-0cb80d03eb36 None None] MFA Rules not processed for user `f3185bd2458c4e22830975700888e9be`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:45.314560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52788c-d5fc-4e6f-9fb5-70481a48dae8 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:45.314848 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52788c-d5fc-4e6f-9fb5-70481a48dae8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:45.315953 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52788c-d5fc-4e6f-9fb5-70481a48dae8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:45.316507 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-9525ae22-a9ac-4e9a-8800-0cb80d03eb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:45.340958 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 811/1634] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Jun 15 14:42:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:45.342568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f65a1-826e-4521-929f-e24d79d74958 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:45.344795 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e52788c-d5fc-4e6f-9fb5-70481a48dae8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:45.345211 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f65a1-826e-4521-929f-e24d79d74958 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:45.345844 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f65a1-826e-4521-929f-e24d79d74958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:45.349299 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e52788c-d5fc-4e6f-9fb5-70481a48dae8 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:45.382266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 824/1635] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:42:45.408537 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-886f65a1-826e-4521-929f-e24d79d74958 None None] MFA Rules not processed for user `e452beb1eaf540daa9bbfec92e3138cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:45.410715 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-886f65a1-826e-4521-929f-e24d79d74958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:45.448685 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52788c-d5fc-4e6f-9fb5-70481a48dae8 service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:45.465064 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52788c-d5fc-4e6f-9fb5-70481a48dae8 service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:45.484030 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ab1035-120c-46f5-810d-a9423132e172 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:45.487778 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab1035-120c-46f5-810d-a9423132e172 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:45.488556 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab1035-120c-46f5-810d-a9423132e172 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:45.489146 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab1035-120c-46f5-810d-a9423132e172 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:45.513370 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ab1035-120c-46f5-810d-a9423132e172 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:45.516213 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ab1035-120c-46f5-810d-a9423132e172 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:45.549497 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 825/1636] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:42:45.610585 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 812/1637] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:45.623222 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4a10c-4226-4a14-84f4-ea87d0f9f160 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:45.624466 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4a10c-4226-4a14-84f4-ea87d0f9f160 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:45.624950 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4a10c-4226-4a14-84f4-ea87d0f9f160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:45.667437 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-dbd4a10c-4226-4a14-84f4-ea87d0f9f160 None None] MFA Rules not processed for user `e452beb1eaf540daa9bbfec92e3138cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:45.674729 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-dbd4a10c-4226-4a14-84f4-ea87d0f9f160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:45.700425 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 826/1638] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:46.445479 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f65a1-826e-4521-929f-e24d79d74958 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:46.463269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f65a1-826e-4521-929f-e24d79d74958 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:46.473675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f0db80-847f-4abf-9874-2840fe73554b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:46.474555 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0db80-847f-4abf-9874-2840fe73554b service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:46.474897 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0db80-847f-4abf-9874-2840fe73554b service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:46.475465 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0db80-847f-4abf-9874-2840fe73554b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:46.635216 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f0db80-847f-4abf-9874-2840fe73554b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:46.636685 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f0db80-847f-4abf-9874-2840fe73554b service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:46.669696 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 813/1639] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:42:46.832064 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd4a10c-4226-4a14-84f4-ea87d0f9f160 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:46.858596 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd4a10c-4226-4a14-84f4-ea87d0f9f160 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:46.869225 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e18e45-8017-4e93-aa86-cfb91b9d00ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:46.871439 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e18e45-8017-4e93-aa86-cfb91b9d00ca service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:46.871801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e18e45-8017-4e93-aa86-cfb91b9d00ca service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:46.872308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e18e45-8017-4e93-aa86-cfb91b9d00ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:46.894448 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e18e45-8017-4e93-aa86-cfb91b9d00ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:46.898276 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e18e45-8017-4e93-aa86-cfb91b9d00ca service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:46.927208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 827/1640] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:42:48.251103 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade2572-58ca-4311-b7dc-816257938271 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:48.251762 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade2572-58ca-4311-b7dc-816257938271 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:48.251762 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade2572-58ca-4311-b7dc-816257938271 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:48.253172 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 814/1641] 10.222.0.45 () {54 vars in 912 bytes} [Wed Jun 15 14:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:42:48.260754 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba4732-4e09-4e90-9bed-cf47f88bedd3 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:48.261277 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba4732-4e09-4e90-9bed-cf47f88bedd3 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:48.261694 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba4732-4e09-4e90-9bed-cf47f88bedd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:48.341175 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-2dba4732-4e09-4e90-9bed-cf47f88bedd3 None None] MFA Rules not processed for user `975ae2790ac849ff88e991231671f510`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:48.342691 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-2dba4732-4e09-4e90-9bed-cf47f88bedd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:48.448200 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 828/1642] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Jun 15 14:42:48 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:48.601621 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ade2572-58ca-4311-b7dc-816257938271 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:48.621291 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ade2572-58ca-4311-b7dc-816257938271 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:48.632615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b1aa71-f795-4237-8b20-ab2febafed57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:48.633649 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1aa71-f795-4237-8b20-ab2febafed57 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:48.634171 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1aa71-f795-4237-8b20-ab2febafed57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:48.634669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1aa71-f795-4237-8b20-ab2febafed57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:48.650687 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b1aa71-f795-4237-8b20-ab2febafed57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:48.653032 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b1aa71-f795-4237-8b20-ab2febafed57 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:48.673571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 815/1643] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:42:49.384065 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba4732-4e09-4e90-9bed-cf47f88bedd3 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:49.529026 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba4732-4e09-4e90-9bed-cf47f88bedd3 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:49.537202 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3731739e-a01b-40e4-9040-5f4d3f048999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:49.543279 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731739e-a01b-40e4-9040-5f4d3f048999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:49.544191 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731739e-a01b-40e4-9040-5f4d3f048999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:49.544879 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731739e-a01b-40e4-9040-5f4d3f048999 admin admin] PATH_INFO: `/v3/users/9202dcd0010f4727b9a4966a5793db78` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:49.551170 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3731739e-a01b-40e4-9040-5f4d3f048999 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9202dcd0010f4727b9a4966a5793db78)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:49.552525 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: WARNING py.warnings [None req-3731739e-a01b-40e4-9040-5f4d3f048999 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:42:49.552525 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: warnings.warn(msg) Jun 15 14:42:49.552525 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: Jun 15 14:42:49.554218 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3731739e-a01b-40e4-9040-5f4d3f048999 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:49.566757 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:42:49.701257 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3731739e-a01b-40e4-9040-5f4d3f048999 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9202dcd0010f4727b9a4966a5793db78'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:42:49.702662 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3731739e-a01b-40e4-9040-5f4d3f048999 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9202dcd0010f4727b9a4966a5793db78'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:42:49.761572 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3731739e-a01b-40e4-9040-5f4d3f048999 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9202dcd0010f4727b9a4966a5793db78'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:42:49.762339 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3731739e-a01b-40e4-9040-5f4d3f048999 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9202dcd0010f4727b9a4966a5793db78'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:42:49.790609 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3731739e-a01b-40e4-9040-5f4d3f048999 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9202dcd0010f4727b9a4966a5793db78'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:42:49.838057 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 829/1644] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:42:49 2022] DELETE /identity/v3/users/9202dcd0010f4727b9a4966a5793db78 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:42:49.844443 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0c741-b061-4720-a527-af3884dab019 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:49.845117 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0c741-b061-4720-a527-af3884dab019 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:49.845117 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0c741-b061-4720-a527-af3884dab019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:49.904405 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-04f0c741-b061-4720-a527-af3884dab019 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:49.905687 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-04f0c741-b061-4720-a527-af3884dab019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:50.173063 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 816/1645] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:50.197416 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3731739e-a01b-40e4-9040-5f4d3f048999 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:50.200881 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-3731739e-a01b-40e4-9040-5f4d3f048999 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:50.529116 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3731739e-a01b-40e4-9040-5f4d3f048999 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:50.544080 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaa3abc-1803-44c6-bb68-49a843073c59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:50.544884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa3abc-1803-44c6-bb68-49a843073c59 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:50.545295 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa3abc-1803-44c6-bb68-49a843073c59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:50.545673 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa3abc-1803-44c6-bb68-49a843073c59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:50.594862 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaa3abc-1803-44c6-bb68-49a843073c59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:50.596921 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaa3abc-1803-44c6-bb68-49a843073c59 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:50.605679 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4025ce8-38fa-45c1-b042-9ff7281e5e8d None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:50.605983 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4025ce8-38fa-45c1-b042-9ff7281e5e8d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:50.606270 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4025ce8-38fa-45c1-b042-9ff7281e5e8d None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:50.608075 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 817/1646] 10.222.0.45 () {54 vars in 960 bytes} [Wed Jun 15 14:42:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:42:50.623570 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a03ad42-380b-4c40-baa9-59b9b972a75d None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:50.623570 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a03ad42-380b-4c40-baa9-59b9b972a75d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:50.623570 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a03ad42-380b-4c40-baa9-59b9b972a75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:50.651958 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-3a03ad42-380b-4c40-baa9-59b9b972a75d None None] MFA Rules not processed for user `f4128111a9aa450abb2146a5d9926dec`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:50.653517 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-3a03ad42-380b-4c40-baa9-59b9b972a75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:50.683826 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 830/1647] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:42:50.691038 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:42:50.938633 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 818/1648] 10.222.0.45 () {58 vars in 1057 bytes} [Wed Jun 15 14:42:50 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:42:50.952837 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaa3abc-1803-44c6-bb68-49a843073c59 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:51.000802 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaa3abc-1803-44c6-bb68-49a843073c59 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:51.029415 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f5672d-414d-4fc0-86ac-a2a717daa599 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4128111a9aa450abb2146a5d9926dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:51.050849 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f5672d-414d-4fc0-86ac-a2a717daa599 service placement] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:51.051607 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f5672d-414d-4fc0-86ac-a2a717daa599 service placement] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:51.051974 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f5672d-414d-4fc0-86ac-a2a717daa599 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:51.058721 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f1f5672d-414d-4fc0-86ac-a2a717daa599 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:51.113840 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f5672d-414d-4fc0-86ac-a2a717daa599 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:51.117082 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f5672d-414d-4fc0-86ac-a2a717daa599 service placement] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:51.151575 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 831/1649] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:42:52.790301 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a03ad42-380b-4c40-baa9-59b9b972a75d None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:52.792032 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-3a03ad42-380b-4c40-baa9-59b9b972a75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:52.840232 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a03ad42-380b-4c40-baa9-59b9b972a75d None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:52.857408 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4e875-86dc-4559-8605-932e4bd490c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:52.861421 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4e875-86dc-4559-8605-932e4bd490c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:52.861714 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4e875-86dc-4559-8605-932e4bd490c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:52.861998 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4e875-86dc-4559-8605-932e4bd490c0 admin admin] PATH_INFO: `/v3/projects/da762c4e79d14510afd0a35251c4bad3` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:52.882250 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4e875-86dc-4559-8605-932e4bd490c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da762c4e79d14510afd0a35251c4bad3)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:52.885041 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: WARNING py.warnings [None req-e6c4e875-86dc-4559-8605-932e4bd490c0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:42:52.885041 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: warnings.warn(msg) Jun 15 14:42:52.885041 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: Jun 15 14:42:52.885955 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4e875-86dc-4559-8605-932e4bd490c0 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:53.119370 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e6c4e875-86dc-4559-8605-932e4bd490c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da762c4e79d14510afd0a35251c4bad3', 'request_id': 'req-e6c4e875-86dc-4559-8605-932e4bd490c0'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:42:53.136841 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e6c4e875-86dc-4559-8605-932e4bd490c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da762c4e79d14510afd0a35251c4bad3', 'request_id': 'req-e6c4e875-86dc-4559-8605-932e4bd490c0'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:42:53.174649 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e6c4e875-86dc-4559-8605-932e4bd490c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da762c4e79d14510afd0a35251c4bad3', 'request_id': 'req-e6c4e875-86dc-4559-8605-932e4bd490c0'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:42:53.216494 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e6c4e875-86dc-4559-8605-932e4bd490c0 admin admin] The token cache is being invalidate because project da762c4e79d14510afd0a35251c4bad3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:42:53.217278 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e6c4e875-86dc-4559-8605-932e4bd490c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:42:53.222452 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e6c4e875-86dc-4559-8605-932e4bd490c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:42:53.230337 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 819/1650] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:42:52 2022] DELETE /identity/v3/projects/da762c4e79d14510afd0a35251c4bad3 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:42:53.236306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bea1f2-0f55-426b-bb2a-d18ec715e52a None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:53.237128 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bea1f2-0f55-426b-bb2a-d18ec715e52a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:53.237590 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bea1f2-0f55-426b-bb2a-d18ec715e52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:53.297082 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-91bea1f2-0f55-426b-bb2a-d18ec715e52a None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:53.298327 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-91bea1f2-0f55-426b-bb2a-d18ec715e52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:53.490708 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 832/1651] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:53.492487 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a10b0-2865-4674-9b83-f9c90ca98ff6 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:53.492784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a10b0-2865-4674-9b83-f9c90ca98ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:53.493073 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a10b0-2865-4674-9b83-f9c90ca98ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:53.549734 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-bd5a10b0-2865-4674-9b83-f9c90ca98ff6 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:53.556907 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-bd5a10b0-2865-4674-9b83-f9c90ca98ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:53.605166 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 820/1652] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:53.605556 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bea1f2-0f55-426b-bb2a-d18ec715e52a None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:53.625014 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bea1f2-0f55-426b-bb2a-d18ec715e52a None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:53.639222 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b399e5e6-9f16-40c3-a3bb-7a476dffc54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:53.641480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399e5e6-9f16-40c3-a3bb-7a476dffc54b admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:53.642257 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399e5e6-9f16-40c3-a3bb-7a476dffc54b admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:53.642257 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399e5e6-9f16-40c3-a3bb-7a476dffc54b admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:53.642961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b399e5e6-9f16-40c3-a3bb-7a476dffc54b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:42:53.643479 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b399e5e6-9f16-40c3-a3bb-7a476dffc54b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:53.646290 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b399e5e6-9f16-40c3-a3bb-7a476dffc54b admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:53.669789 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:42:53.681477 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 833/1653] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:42:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:42:53.686045 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a10b0-2865-4674-9b83-f9c90ca98ff6 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:53.707385 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a10b0-2865-4674-9b83-f9c90ca98ff6 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:53.714755 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ddbe27-3d92-431c-8ad2-5fc4d6c1f74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:53.714964 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddbe27-3d92-431c-8ad2-5fc4d6c1f74f admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:53.716102 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddbe27-3d92-431c-8ad2-5fc4d6c1f74f admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:53.716102 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddbe27-3d92-431c-8ad2-5fc4d6c1f74f admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:53.716840 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ddbe27-3d92-431c-8ad2-5fc4d6c1f74f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:53.719488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ddbe27-3d92-431c-8ad2-5fc4d6c1f74f admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:53.830074 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 821/1654] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:42:53 2022] POST /identity/v3/projects => generated 363 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:42:53.838598 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b399e5e6-9f16-40c3-a3bb-7a476dffc54b admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:53.857177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b399e5e6-9f16-40c3-a3bb-7a476dffc54b admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:54.058603 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e72246-343b-457c-897a-dc98ac2cce4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:54.059619 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e72246-343b-457c-897a-dc98ac2cce4e admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:54.060345 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e72246-343b-457c-897a-dc98ac2cce4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:54.062636 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e72246-343b-457c-897a-dc98ac2cce4e admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:54.064034 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e72246-343b-457c-897a-dc98ac2cce4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:54.066236 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e72246-343b-457c-897a-dc98ac2cce4e admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:54.122493 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 834/1655] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:42:53 2022] POST /identity/v3/users => generated 282 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:42:54.126241 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ddbe27-3d92-431c-8ad2-5fc4d6c1f74f admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:54.145508 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ddbe27-3d92-431c-8ad2-5fc4d6c1f74f admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:54.153319 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5a17e-fa9c-4491-b008-6592834deea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:54.154908 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5a17e-fa9c-4491-b008-6592834deea9 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:54.155502 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5a17e-fa9c-4491-b008-6592834deea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:54.155935 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5a17e-fa9c-4491-b008-6592834deea9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:54.156886 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f5a17e-fa9c-4491-b008-6592834deea9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:54.162109 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f5a17e-fa9c-4491-b008-6592834deea9 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:54.203072 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 822/1656] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:42:54.205829 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e72246-343b-457c-897a-dc98ac2cce4e admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:54.221953 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e72246-343b-457c-897a-dc98ac2cce4e admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:54.232509 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118f7ad-8ef7-4c01-822a-6e4a5ce00597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:54.235648 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118f7ad-8ef7-4c01-822a-6e4a5ce00597 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:54.236901 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118f7ad-8ef7-4c01-822a-6e4a5ce00597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:54.238750 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118f7ad-8ef7-4c01-822a-6e4a5ce00597 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:54.242024 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4118f7ad-8ef7-4c01-822a-6e4a5ce00597 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:54.245841 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4118f7ad-8ef7-4c01-822a-6e4a5ce00597 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:54.275792 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 835/1657] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:42:54.280079 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5a17e-fa9c-4491-b008-6592834deea9 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:54.292052 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5a17e-fa9c-4491-b008-6592834deea9 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:54.324216 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce7ea34-a354-4c34-a275-9451919d1791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:54.325457 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7ea34-a354-4c34-a275-9451919d1791 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:54.325748 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7ea34-a354-4c34-a275-9451919d1791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:54.326482 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7ea34-a354-4c34-a275-9451919d1791 admin admin] PATH_INFO: `/v3/projects/5c754fd650de45f69e4aa610d7f4b47d/users/bdd1f2da42d44eb88257b56fb5561d6e/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:54.376848 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce7ea34-a354-4c34-a275-9451919d1791 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c754fd650de45f69e4aa610d7f4b47d, user_id=bdd1f2da42d44eb88257b56fb5561d6e, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:54.379719 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce7ea34-a354-4c34-a275-9451919d1791 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:54.419046 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 823/1658] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:42:54 2022] PUT /identity/v3/projects/5c754fd650de45f69e4aa610d7f4b47d/users/bdd1f2da42d44eb88257b56fb5561d6e/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:42:54.433256 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98e4e5-7b7c-4b4d-aa9a-5593ac1bc195 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:54.433736 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98e4e5-7b7c-4b4d-aa9a-5593ac1bc195 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:54.435195 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98e4e5-7b7c-4b4d-aa9a-5593ac1bc195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:54.524559 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-0c98e4e5-7b7c-4b4d-aa9a-5593ac1bc195 None None] MFA Rules not processed for user `bdd1f2da42d44eb88257b56fb5561d6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:54.526629 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-0c98e4e5-7b7c-4b4d-aa9a-5593ac1bc195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:54.720041 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 836/1659] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:54 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:42:54.720587 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce7ea34-a354-4c34-a275-9451919d1791 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:54.790142 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:42:54.881696 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce7ea34-a354-4c34-a275-9451919d1791 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:54.897304 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e8b83-6117-4507-a13d-916127224968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:54.898295 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e8b83-6117-4507-a13d-916127224968 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:54.898721 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e8b83-6117-4507-a13d-916127224968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:54.899171 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e8b83-6117-4507-a13d-916127224968 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:54.900038 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37e8b83-6117-4507-a13d-916127224968 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:54.903911 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37e8b83-6117-4507-a13d-916127224968 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:54.963289 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 824/1660] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:42:54 2022] POST /identity/v3/projects => generated 365 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:42:54.969005 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c98e4e5-7b7c-4b4d-aa9a-5593ac1bc195 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:54.993444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c98e4e5-7b7c-4b4d-aa9a-5593ac1bc195 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:55.136425 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13654305-4865-462e-938f-ca959ae01fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:55.137321 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13654305-4865-462e-938f-ca959ae01fdd admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:55.137766 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13654305-4865-462e-938f-ca959ae01fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:55.138146 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13654305-4865-462e-938f-ca959ae01fdd admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:55.138979 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13654305-4865-462e-938f-ca959ae01fdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:55.142803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13654305-4865-462e-938f-ca959ae01fdd admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:55.175074 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:42:55.202621 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e8b83-6117-4507-a13d-916127224968 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:55.203443 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 837/1661] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:42:54 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:42:55.223696 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e8b83-6117-4507-a13d-916127224968 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:55.243358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a97d42-e6e2-4f80-85ed-b71f703cc5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:55.244244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a97d42-e6e2-4f80-85ed-b71f703cc5ff admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:55.244579 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a97d42-e6e2-4f80-85ed-b71f703cc5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:55.244911 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a97d42-e6e2-4f80-85ed-b71f703cc5ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:55.245780 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a97d42-e6e2-4f80-85ed-b71f703cc5ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:55.247361 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a97d42-e6e2-4f80-85ed-b71f703cc5ff admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:55.266053 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 825/1662] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:42:55.273359 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13654305-4865-462e-938f-ca959ae01fdd admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:55.297771 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13654305-4865-462e-938f-ca959ae01fdd admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:55.327023 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd8ffc-6bc4-4a5f-8bf7-118a49e3c215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:55.339751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd8ffc-6bc4-4a5f-8bf7-118a49e3c215 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:55.340304 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd8ffc-6bc4-4a5f-8bf7-118a49e3c215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:55.340773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd8ffc-6bc4-4a5f-8bf7-118a49e3c215 admin admin] PATH_INFO: `/v3/projects/48b41c830eb2403f8b3067e220af3589/users/d6f8124cd8db4c19990be3fca7045b14/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:55.382889 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd8ffc-6bc4-4a5f-8bf7-118a49e3c215 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48b41c830eb2403f8b3067e220af3589, user_id=d6f8124cd8db4c19990be3fca7045b14, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:55.392272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd8ffc-6bc4-4a5f-8bf7-118a49e3c215 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:55.425161 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 838/1663] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:42:55 2022] PUT /identity/v3/projects/48b41c830eb2403f8b3067e220af3589/users/d6f8124cd8db4c19990be3fca7045b14/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:42:55.429890 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a97d42-e6e2-4f80-85ed-b71f703cc5ff admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:55.449550 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a97d42-e6e2-4f80-85ed-b71f703cc5ff admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:55.718369 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a439620-afe5-4616-bdc5-aa1b602f680d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:55.720067 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a439620-afe5-4616-bdc5-aa1b602f680d admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:55.720315 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a439620-afe5-4616-bdc5-aa1b602f680d admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:55.735505 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a439620-afe5-4616-bdc5-aa1b602f680d admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:55.736582 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a439620-afe5-4616-bdc5-aa1b602f680d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:55.738167 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a439620-afe5-4616-bdc5-aa1b602f680d admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:55.766139 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd8ffc-6bc4-4a5f-8bf7-118a49e3c215 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:55.767317 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 826/1664] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:42:55.807093 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd8ffc-6bc4-4a5f-8bf7-118a49e3c215 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:55.807093 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7cfade-5dda-422f-b20a-21889806793e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:55.807093 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7cfade-5dda-422f-b20a-21889806793e admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:55.807093 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7cfade-5dda-422f-b20a-21889806793e admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:55.808566 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7cfade-5dda-422f-b20a-21889806793e admin admin] PATH_INFO: `/v3/projects/48b41c830eb2403f8b3067e220af3589/users/d6f8124cd8db4c19990be3fca7045b14/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:55.808566 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7cfade-5dda-422f-b20a-21889806793e admin admin] RBAC: Authorizing `identity:create_grant(project_id=48b41c830eb2403f8b3067e220af3589, user_id=d6f8124cd8db4c19990be3fca7045b14, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:55.811604 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397e083-ebd8-4d73-82fb-5511aedb919b None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:55.812732 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397e083-ebd8-4d73-82fb-5511aedb919b None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:55.813124 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397e083-ebd8-4d73-82fb-5511aedb919b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:55.829123 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7cfade-5dda-422f-b20a-21889806793e admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:55.867323 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 839/1665] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:42:55 2022] PUT /identity/v3/projects/48b41c830eb2403f8b3067e220af3589/users/d6f8124cd8db4c19990be3fca7045b14/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:42:55.883142 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b09c34-46bf-40e8-a902-b0f1c0b26aa2 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:55.883532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b09c34-46bf-40e8-a902-b0f1c0b26aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:55.884008 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b09c34-46bf-40e8-a902-b0f1c0b26aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:55.903187 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-b397e083-ebd8-4d73-82fb-5511aedb919b None None] MFA Rules not processed for user `c9a3a2e49fee4ea08ca790006d0b29e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:55.906386 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b397e083-ebd8-4d73-82fb-5511aedb919b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:55.964260 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-d3b09c34-46bf-40e8-a902-b0f1c0b26aa2 None None] MFA Rules not processed for user `d6f8124cd8db4c19990be3fca7045b14`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:55.966233 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d3b09c34-46bf-40e8-a902-b0f1c0b26aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:56.115023 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 827/1666] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:42:56.128471 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e35cce-b78f-46aa-85d3-7e86dd6724d2 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:56.130307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e35cce-b78f-46aa-85d3-7e86dd6724d2 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:56.134197 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e35cce-b78f-46aa-85d3-7e86dd6724d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:56.184440 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-33e35cce-b78f-46aa-85d3-7e86dd6724d2 None None] MFA Rules not processed for user `c9a3a2e49fee4ea08ca790006d0b29e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:56.186406 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-33e35cce-b78f-46aa-85d3-7e86dd6724d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:56.198394 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 840/1667] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:55 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:56.204767 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b09c34-46bf-40e8-a902-b0f1c0b26aa2 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:56.208675 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d3b09c34-46bf-40e8-a902-b0f1c0b26aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:56.233756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 828/1668] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:42:56.245138 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e35cce-b78f-46aa-85d3-7e86dd6724d2 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:56.246823 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-33e35cce-b78f-46aa-85d3-7e86dd6724d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:56.297645 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:42:56.455464 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b09c34-46bf-40e8-a902-b0f1c0b26aa2 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:56.463793 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50597182-8d57-44f0-9416-f14538d33c71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:56.464618 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50597182-8d57-44f0-9416-f14538d33c71 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:56.465055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50597182-8d57-44f0-9416-f14538d33c71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:56.465334 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50597182-8d57-44f0-9416-f14538d33c71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:56.512656 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50597182-8d57-44f0-9416-f14538d33c71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:56.515104 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50597182-8d57-44f0-9416-f14538d33c71 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:56.563055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 841/1669] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:56 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:42:56.563566 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e35cce-b78f-46aa-85d3-7e86dd6724d2 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:56.604052 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dded52-af2e-4e33-8f4d-a23ed3180a25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:56.605266 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dded52-af2e-4e33-8f4d-a23ed3180a25 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:56.605746 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dded52-af2e-4e33-8f4d-a23ed3180a25 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:56.608766 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dded52-af2e-4e33-8f4d-a23ed3180a25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:56.649908 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dded52-af2e-4e33-8f4d-a23ed3180a25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:56.653074 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dded52-af2e-4e33-8f4d-a23ed3180a25 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:56.739117 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 829/1670] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:42:56.808692 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50597182-8d57-44f0-9416-f14538d33c71 service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:56.810451 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-50597182-8d57-44f0-9416-f14538d33c71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:56.854865 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50597182-8d57-44f0-9416-f14538d33c71 service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:56.867103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffff2ac-2ee5-4344-9f4c-a6f23d715478 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:56.867963 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffff2ac-2ee5-4344-9f4c-a6f23d715478 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:56.868361 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffff2ac-2ee5-4344-9f4c-a6f23d715478 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:56.868741 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffff2ac-2ee5-4344-9f4c-a6f23d715478 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:57.074428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:42:57.096334 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffff2ac-2ee5-4344-9f4c-a6f23d715478 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:57.098094 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffff2ac-2ee5-4344-9f4c-a6f23d715478 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:57.132420 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 842/1671] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:56 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:42:57.867239 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eefe58-aa4a-438b-b7e7-ef7b99df1abd None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:57.867935 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eefe58-aa4a-438b-b7e7-ef7b99df1abd None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:57.867935 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eefe58-aa4a-438b-b7e7-ef7b99df1abd None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:57.870048 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 830/1672] 10.222.0.45 () {54 vars in 912 bytes} [Wed Jun 15 14:42:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:42:57.899869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e7f85-ad60-4275-b104-0a455b3db81d None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:57.900635 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e7f85-ad60-4275-b104-0a455b3db81d None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:57.900714 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e7f85-ad60-4275-b104-0a455b3db81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:57.913631 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68508c37-354f-44f1-bc15-95449cb4c7e0 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:57.916194 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68508c37-354f-44f1-bc15-95449cb4c7e0 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:57.922525 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68508c37-354f-44f1-bc15-95449cb4c7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:57.966300 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-094e7f85-ad60-4275-b104-0a455b3db81d None None] MFA Rules not processed for user `1efb0951ba1144d58698e02e0e5e9416`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:57.967746 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-094e7f85-ad60-4275-b104-0a455b3db81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:57.990963 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-68508c37-354f-44f1-bc15-95449cb4c7e0 None None] MFA Rules not processed for user `975ae2790ac849ff88e991231671f510`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:57.993436 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-68508c37-354f-44f1-bc15-95449cb4c7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:58.103613 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 843/1673] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:58.123499 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e7f85-ad60-4275-b104-0a455b3db81d None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:58.131974 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-094e7f85-ad60-4275-b104-0a455b3db81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:58.153344 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 831/1674] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Jun 15 14:42:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:58.160120 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68508c37-354f-44f1-bc15-95449cb4c7e0 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:58.161514 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-68508c37-354f-44f1-bc15-95449cb4c7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:58.166076 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e7f85-ad60-4275-b104-0a455b3db81d None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:58.174480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe28be0-2c15-4f2b-9352-c7f938eaa8fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:58.175184 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe28be0-2c15-4f2b-9352-c7f938eaa8fe service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:58.175371 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe28be0-2c15-4f2b-9352-c7f938eaa8fe service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:58.175623 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe28be0-2c15-4f2b-9352-c7f938eaa8fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:58.186504 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68508c37-354f-44f1-bc15-95449cb4c7e0 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:58.194092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b989af-853c-440e-9c43-fb9d8c6988f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:58.196612 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b989af-853c-440e-9c43-fb9d8c6988f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:58.197179 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b989af-853c-440e-9c43-fb9d8c6988f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:58.197657 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b989af-853c-440e-9c43-fb9d8c6988f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:58.198958 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe28be0-2c15-4f2b-9352-c7f938eaa8fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:58.200894 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe28be0-2c15-4f2b-9352-c7f938eaa8fe service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:58.217947 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b989af-853c-440e-9c43-fb9d8c6988f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:58.220374 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b989af-853c-440e-9c43-fb9d8c6988f6 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:58.233777 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 844/1675] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:42:58.256344 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 832/1676] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:42:58.955435 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d71ad2a-2019-4e08-935c-848f51bc7192 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:58.955889 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d71ad2a-2019-4e08-935c-848f51bc7192 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:58.955986 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d71ad2a-2019-4e08-935c-848f51bc7192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:59.019572 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-9d71ad2a-2019-4e08-935c-848f51bc7192 None None] MFA Rules not processed for user `25ba370b8daf49e48514cd49c2851215`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:59.022269 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-9d71ad2a-2019-4e08-935c-848f51bc7192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:59.235852 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 845/1677] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:59.247524 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b989af-853c-440e-9c43-fb9d8c6988f6 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:59.331146 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b989af-853c-440e-9c43-fb9d8c6988f6 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:59.351901 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334ae321-e248-460f-bf3f-2f8fda71b5ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:59.354225 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ae321-e248-460f-bf3f-2f8fda71b5ff service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:59.356083 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ae321-e248-460f-bf3f-2f8fda71b5ff service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:59.366752 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ae321-e248-460f-bf3f-2f8fda71b5ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:59.404533 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55945e0-c33d-41ef-aa48-10eb36ea19f7 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:59.406665 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55945e0-c33d-41ef-aa48-10eb36ea19f7 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:59.407996 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55945e0-c33d-41ef-aa48-10eb36ea19f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:59.438571 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-e55945e0-c33d-41ef-aa48-10eb36ea19f7 None None] MFA Rules not processed for user `c9a3a2e49fee4ea08ca790006d0b29e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:59.440654 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e55945e0-c33d-41ef-aa48-10eb36ea19f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:59.445368 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334ae321-e248-460f-bf3f-2f8fda71b5ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:59.453152 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334ae321-e248-460f-bf3f-2f8fda71b5ff service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:59.462412 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 846/1678] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:42:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:59.489586 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 833/1679] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:02.729926 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a47c7-a5d0-4917-bc6c-5dc414644b06 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:02.730503 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a47c7-a5d0-4917-bc6c-5dc414644b06 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:02.730642 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a47c7-a5d0-4917-bc6c-5dc414644b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:02.820361 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-981a47c7-a5d0-4917-bc6c-5dc414644b06 None None] MFA Rules not processed for user `bf393488726e49338e70017f5cb7c4be`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:02.822428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-981a47c7-a5d0-4917-bc6c-5dc414644b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:03.105463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:03.195624 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 847/1680] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Jun 15 14:43:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:03.522873 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334ae321-e248-460f-bf3f-2f8fda71b5ff service nova] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:03.525794 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-334ae321-e248-460f-bf3f-2f8fda71b5ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:03.841191 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334ae321-e248-460f-bf3f-2f8fda71b5ff service nova] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:03.850418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc62b6-adca-4b07-b66d-d18f97ec0bda service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '641a81b3140f47119b5c2323a97a6681', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:03.851139 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc62b6-adca-4b07-b66d-d18f97ec0bda service swift] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:03.851423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc62b6-adca-4b07-b66d-d18f97ec0bda service swift] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:03.851856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc62b6-adca-4b07-b66d-d18f97ec0bda service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:03.887590 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fc62b6-adca-4b07-b66d-d18f97ec0bda service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:03.901390 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fc62b6-adca-4b07-b66d-d18f97ec0bda service swift] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:03.952332 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 834/1681] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Jun 15 14:43:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 15 14:43:04.148856 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a47c7-a5d0-4917-bc6c-5dc414644b06 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:04.662969 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a47c7-a5d0-4917-bc6c-5dc414644b06 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:04.690468 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0afded9-b6fe-4870-a77f-a1182c77913f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:04.697782 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afded9-b6fe-4870-a77f-a1182c77913f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:04.697782 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afded9-b6fe-4870-a77f-a1182c77913f admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:04.699246 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afded9-b6fe-4870-a77f-a1182c77913f admin admin] PATH_INFO: `/v3/users/bdd1f2da42d44eb88257b56fb5561d6e` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:04.709124 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0afded9-b6fe-4870-a77f-a1182c77913f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdd1f2da42d44eb88257b56fb5561d6e)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:04.719511 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0afded9-b6fe-4870-a77f-a1182c77913f admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:04.814505 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211bf760-76f6-413b-b238-b96c4c5f8b9d None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:04.815839 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211bf760-76f6-413b-b238-b96c4c5f8b9d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:04.816438 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211bf760-76f6-413b-b238-b96c4c5f8b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:04.879164 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-211bf760-76f6-413b-b238-b96c4c5f8b9d None None] MFA Rules not processed for user `1efb0951ba1144d58698e02e0e5e9416`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:04.881024 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-211bf760-76f6-413b-b238-b96c4c5f8b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:04.918159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d0afded9-b6fe-4870-a77f-a1182c77913f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdd1f2da42d44eb88257b56fb5561d6e'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:04.924438 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d0afded9-b6fe-4870-a77f-a1182c77913f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdd1f2da42d44eb88257b56fb5561d6e'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:04.958392 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 835/1682] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:05.035926 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d0afded9-b6fe-4870-a77f-a1182c77913f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdd1f2da42d44eb88257b56fb5561d6e'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:05.038351 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d0afded9-b6fe-4870-a77f-a1182c77913f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdd1f2da42d44eb88257b56fb5561d6e'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:05.070272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d0afded9-b6fe-4870-a77f-a1182c77913f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdd1f2da42d44eb88257b56fb5561d6e'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:05.153294 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 848/1683] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:04 2022] DELETE /identity/v3/users/bdd1f2da42d44eb88257b56fb5561d6e => generated 0 bytes in 1005 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:05.157018 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ecc6b-8acd-4a66-b63c-db824cb98f24 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:05.157720 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ecc6b-8acd-4a66-b63c-db824cb98f24 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:05.158185 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ecc6b-8acd-4a66-b63c-db824cb98f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:05.208328 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-d53ecc6b-8acd-4a66-b63c-db824cb98f24 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:05.216705 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d53ecc6b-8acd-4a66-b63c-db824cb98f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:05.242183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:05.490784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 836/1684] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:05.668856 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0afded9-b6fe-4870-a77f-a1182c77913f admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:05.670814 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d0afded9-b6fe-4870-a77f-a1182c77913f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:05.911039 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0afded9-b6fe-4870-a77f-a1182c77913f admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:05.936101 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4622c048-4d09-4bae-be44-020428deb464 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:05.936915 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622c048-4d09-4bae-be44-020428deb464 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:05.937338 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622c048-4d09-4bae-be44-020428deb464 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:05.937640 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622c048-4d09-4bae-be44-020428deb464 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:05.998232 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4622c048-4d09-4bae-be44-020428deb464 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:06.004102 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4622c048-4d09-4bae-be44-020428deb464 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:06.058391 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 849/1685] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:06.110740 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ecc6b-8acd-4a66-b63c-db824cb98f24 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:06.112207 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d53ecc6b-8acd-4a66-b63c-db824cb98f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:06.155518 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ecc6b-8acd-4a66-b63c-db824cb98f24 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:06.190195 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae60e4-8b2e-43d9-8b01-02eab6298e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:06.191719 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae60e4-8b2e-43d9-8b01-02eab6298e8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:06.192362 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae60e4-8b2e-43d9-8b01-02eab6298e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:06.192958 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae60e4-8b2e-43d9-8b01-02eab6298e8a admin admin] PATH_INFO: `/v3/users/627ebb2f052145cb8a009e1009d15da7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:06.198268 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae60e4-8b2e-43d9-8b01-02eab6298e8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=627ebb2f052145cb8a009e1009d15da7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:06.203289 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae60e4-8b2e-43d9-8b01-02eab6298e8a admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:06.453168 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-bfae60e4-8b2e-43d9-8b01-02eab6298e8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '627ebb2f052145cb8a009e1009d15da7'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:06.456093 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-bfae60e4-8b2e-43d9-8b01-02eab6298e8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '627ebb2f052145cb8a009e1009d15da7'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:06.504402 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-bfae60e4-8b2e-43d9-8b01-02eab6298e8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '627ebb2f052145cb8a009e1009d15da7'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:06.520033 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-bfae60e4-8b2e-43d9-8b01-02eab6298e8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '627ebb2f052145cb8a009e1009d15da7'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:06.534343 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-bfae60e4-8b2e-43d9-8b01-02eab6298e8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '627ebb2f052145cb8a009e1009d15da7'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:06.556330 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 837/1686] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:06 2022] DELETE /identity/v3/users/627ebb2f052145cb8a009e1009d15da7 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:06.557332 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fbcaa-9f29-4f32-9c20-6fdf41954833 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:06.558117 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fbcaa-9f29-4f32-9c20-6fdf41954833 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:06.559324 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fbcaa-9f29-4f32-9c20-6fdf41954833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:06.634808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-1e0fbcaa-9f29-4f32-9c20-6fdf41954833 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:06.636149 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1e0fbcaa-9f29-4f32-9c20-6fdf41954833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:06.673655 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:06.946733 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 850/1687] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:07.099732 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae60e4-8b2e-43d9-8b01-02eab6298e8a admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:07.108054 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-bfae60e4-8b2e-43d9-8b01-02eab6298e8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:07.404391 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae60e4-8b2e-43d9-8b01-02eab6298e8a admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:07.411187 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1286a-8d1e-4bca-956b-86b51bfe79ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:07.412909 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1286a-8d1e-4bca-956b-86b51bfe79ee service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:07.412909 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1286a-8d1e-4bca-956b-86b51bfe79ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:07.413226 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1286a-8d1e-4bca-956b-86b51bfe79ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:07.433995 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d1286a-8d1e-4bca-956b-86b51bfe79ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:07.436972 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d1286a-8d1e-4bca-956b-86b51bfe79ee service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:07.470110 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 838/1688] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:08.617237 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e768a-8343-4b46-9165-0940a3f937b4 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:08.618856 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e768a-8343-4b46-9165-0940a3f937b4 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:08.619588 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e768a-8343-4b46-9165-0940a3f937b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:08.757177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-048e768a-8343-4b46-9165-0940a3f937b4 None None] MFA Rules not processed for user `975ae2790ac849ff88e991231671f510`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:08.759686 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-048e768a-8343-4b46-9165-0940a3f937b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:08.794074 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 851/1689] 10.222.0.47 () {58 vars in 1015 bytes} [Wed Jun 15 14:43:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:08.856121 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1286a-8d1e-4bca-956b-86b51bfe79ee service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:08.860120 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-08d1286a-8d1e-4bca-956b-86b51bfe79ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:08.929339 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1286a-8d1e-4bca-956b-86b51bfe79ee service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:08.937796 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5551b1-3170-47db-a5ac-b0bc4de44abf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:08.942803 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5551b1-3170-47db-a5ac-b0bc4de44abf service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:08.944216 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5551b1-3170-47db-a5ac-b0bc4de44abf service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:08.945013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5551b1-3170-47db-a5ac-b0bc4de44abf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:08.986749 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5551b1-3170-47db-a5ac-b0bc4de44abf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:08.998412 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5551b1-3170-47db-a5ac-b0bc4de44abf service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:09.039123 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 839/1690] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:10.696256 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e768a-8343-4b46-9165-0940a3f937b4 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:10.697816 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-048e768a-8343-4b46-9165-0940a3f937b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:10.743912 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e768a-8343-4b46-9165-0940a3f937b4 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:10.759560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8060d15-c285-4632-9ee8-bc3a2d9d1ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:10.760676 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8060d15-c285-4632-9ee8-bc3a2d9d1ebc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:10.761209 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8060d15-c285-4632-9ee8-bc3a2d9d1ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:10.761728 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8060d15-c285-4632-9ee8-bc3a2d9d1ebc admin admin] PATH_INFO: `/v3/projects/5c754fd650de45f69e4aa610d7f4b47d` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:10.764650 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8060d15-c285-4632-9ee8-bc3a2d9d1ebc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c754fd650de45f69e4aa610d7f4b47d)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:10.770285 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8060d15-c285-4632-9ee8-bc3a2d9d1ebc admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:10.911082 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f8060d15-c285-4632-9ee8-bc3a2d9d1ebc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c754fd650de45f69e4aa610d7f4b47d', 'request_id': 'req-f8060d15-c285-4632-9ee8-bc3a2d9d1ebc'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:10.927945 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f8060d15-c285-4632-9ee8-bc3a2d9d1ebc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c754fd650de45f69e4aa610d7f4b47d', 'request_id': 'req-f8060d15-c285-4632-9ee8-bc3a2d9d1ebc'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:10.950698 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f8060d15-c285-4632-9ee8-bc3a2d9d1ebc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c754fd650de45f69e4aa610d7f4b47d', 'request_id': 'req-f8060d15-c285-4632-9ee8-bc3a2d9d1ebc'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:11.000442 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f8060d15-c285-4632-9ee8-bc3a2d9d1ebc admin admin] The token cache is being invalidate because project 5c754fd650de45f69e4aa610d7f4b47d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:11.000969 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f8060d15-c285-4632-9ee8-bc3a2d9d1ebc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:11.002192 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f8060d15-c285-4632-9ee8-bc3a2d9d1ebc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:11.005277 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 852/1691] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:10 2022] DELETE /identity/v3/projects/5c754fd650de45f69e4aa610d7f4b47d => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:43:11.010213 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5551b1-3170-47db-a5ac-b0bc4de44abf service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:11.031712 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5551b1-3170-47db-a5ac-b0bc4de44abf service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:11.066933 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:11.181740 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4743f71-30e1-49a1-9282-051cb6d5b9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:11.183312 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4743f71-30e1-49a1-9282-051cb6d5b9b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:11.183891 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4743f71-30e1-49a1-9282-051cb6d5b9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:11.184362 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4743f71-30e1-49a1-9282-051cb6d5b9b1 admin admin] PATH_INFO: `/v3/users/d6f8124cd8db4c19990be3fca7045b14` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:11.187630 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4743f71-30e1-49a1-9282-051cb6d5b9b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6f8124cd8db4c19990be3fca7045b14)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:11.190147 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4743f71-30e1-49a1-9282-051cb6d5b9b1 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:11.315985 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c4743f71-30e1-49a1-9282-051cb6d5b9b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6f8124cd8db4c19990be3fca7045b14'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:11.318677 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c4743f71-30e1-49a1-9282-051cb6d5b9b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6f8124cd8db4c19990be3fca7045b14'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:11.360146 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c4743f71-30e1-49a1-9282-051cb6d5b9b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6f8124cd8db4c19990be3fca7045b14'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:11.382084 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c4743f71-30e1-49a1-9282-051cb6d5b9b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6f8124cd8db4c19990be3fca7045b14'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:11.432637 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c4743f71-30e1-49a1-9282-051cb6d5b9b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6f8124cd8db4c19990be3fca7045b14'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:11.466154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 840/1692] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:11 2022] DELETE /identity/v3/users/d6f8124cd8db4c19990be3fca7045b14 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:12.721536 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844c139-a4c5-4712-add0-041198036426 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:12.737398 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844c139-a4c5-4712-add0-041198036426 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:12.740071 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844c139-a4c5-4712-add0-041198036426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:12.878654 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-c844c139-a4c5-4712-add0-041198036426 None None] MFA Rules not processed for user `25ba370b8daf49e48514cd49c2851215`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:12.880871 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c844c139-a4c5-4712-add0-041198036426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:13.081853 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 853/1693] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:14.210327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850aaa00-fdfd-4173-bf8f-9f6b6f8b7d1a None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:14.211990 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850aaa00-fdfd-4173-bf8f-9f6b6f8b7d1a None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:14.213150 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850aaa00-fdfd-4173-bf8f-9f6b6f8b7d1a None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:14.216660 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 841/1694] 10.222.0.45 () {54 vars in 921 bytes} [Wed Jun 15 14:43:14 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:43:14.334548 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844c139-a4c5-4712-add0-041198036426 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:14.337324 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c844c139-a4c5-4712-add0-041198036426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:14.383427 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850aaa00-fdfd-4173-bf8f-9f6b6f8b7d1a None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:14.389696 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-850aaa00-fdfd-4173-bf8f-9f6b6f8b7d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:14.623911 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:14.712334 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844c139-a4c5-4712-add0-041198036426 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:14.734842 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab5a5c-1641-45a5-a6f9-b3bb39e27168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:14.736161 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab5a5c-1641-45a5-a6f9-b3bb39e27168 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:14.738327 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab5a5c-1641-45a5-a6f9-b3bb39e27168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:14.738863 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab5a5c-1641-45a5-a6f9-b3bb39e27168 admin admin] PATH_INFO: `/v3/projects/c1f8a1b95df743588b298fe0efbcd967` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:14.744244 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab5a5c-1641-45a5-a6f9-b3bb39e27168 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1f8a1b95df743588b298fe0efbcd967)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:14.755035 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab5a5c-1641-45a5-a6f9-b3bb39e27168 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:14.829164 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850aaa00-fdfd-4173-bf8f-9f6b6f8b7d1a None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:14.862958 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85876af-1db9-41a2-afc6-745fd21b360d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:14.865763 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85876af-1db9-41a2-afc6-745fd21b360d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:14.867015 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85876af-1db9-41a2-afc6-745fd21b360d admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:14.868647 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85876af-1db9-41a2-afc6-745fd21b360d admin admin] PATH_INFO: `/v3/users/98c07f9a7da44055959709fd84f3042e` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:14.870294 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85876af-1db9-41a2-afc6-745fd21b360d admin admin] RBAC: Authorizing `identity:delete_user(user_id=98c07f9a7da44055959709fd84f3042e)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:14.873141 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85876af-1db9-41a2-afc6-745fd21b360d admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:14.906778 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-11ab5a5c-1641-45a5-a6f9-b3bb39e27168 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1f8a1b95df743588b298fe0efbcd967', 'request_id': 'req-11ab5a5c-1641-45a5-a6f9-b3bb39e27168'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:14.926447 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-11ab5a5c-1641-45a5-a6f9-b3bb39e27168 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1f8a1b95df743588b298fe0efbcd967', 'request_id': 'req-11ab5a5c-1641-45a5-a6f9-b3bb39e27168'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:14.946534 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-11ab5a5c-1641-45a5-a6f9-b3bb39e27168 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1f8a1b95df743588b298fe0efbcd967', 'request_id': 'req-11ab5a5c-1641-45a5-a6f9-b3bb39e27168'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:14.970891 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:14.971362 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-11ab5a5c-1641-45a5-a6f9-b3bb39e27168 admin admin] The token cache is being invalidate because project c1f8a1b95df743588b298fe0efbcd967 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:14.971765 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-11ab5a5c-1641-45a5-a6f9-b3bb39e27168 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:14.973463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-11ab5a5c-1641-45a5-a6f9-b3bb39e27168 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:14.976978 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 854/1695] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:14 2022] DELETE /identity/v3/projects/c1f8a1b95df743588b298fe0efbcd967 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:14.977998 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab5a5c-1641-45a5-a6f9-b3bb39e27168 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:14.979792 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-11ab5a5c-1641-45a5-a6f9-b3bb39e27168 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:15.004285 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c85876af-1db9-41a2-afc6-745fd21b360d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98c07f9a7da44055959709fd84f3042e'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:15.005000 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c85876af-1db9-41a2-afc6-745fd21b360d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98c07f9a7da44055959709fd84f3042e'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:15.037282 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c85876af-1db9-41a2-afc6-745fd21b360d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98c07f9a7da44055959709fd84f3042e'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:15.056761 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c85876af-1db9-41a2-afc6-745fd21b360d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98c07f9a7da44055959709fd84f3042e'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:15.093360 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c85876af-1db9-41a2-afc6-745fd21b360d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98c07f9a7da44055959709fd84f3042e'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:15.112764 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 842/1696] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:14 2022] DELETE /identity/v3/users/98c07f9a7da44055959709fd84f3042e => generated 0 bytes in 730 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:15.114618 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ec53a-9a27-4c93-b81e-df5f5a1bcc6c None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:15.115593 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ec53a-9a27-4c93-b81e-df5f5a1bcc6c None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:15.115784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ec53a-9a27-4c93-b81e-df5f5a1bcc6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:15.179860 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-ca7ec53a-9a27-4c93-b81e-df5f5a1bcc6c None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:15.181236 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ca7ec53a-9a27-4c93-b81e-df5f5a1bcc6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:15.194163 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab5a5c-1641-45a5-a6f9-b3bb39e27168 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:15.434237 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 843/1697] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:15.440938 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92724c-79c2-4d93-b131-2619bec8a673 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:15.441788 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92724c-79c2-4d93-b131-2619bec8a673 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:15.444513 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92724c-79c2-4d93-b131-2619bec8a673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:15.449231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf47ef-e7b0-4260-9ba7-0f06aa65d61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:15.451769 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf47ef-e7b0-4260-9ba7-0f06aa65d61b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:15.452061 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf47ef-e7b0-4260-9ba7-0f06aa65d61b admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:15.452410 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf47ef-e7b0-4260-9ba7-0f06aa65d61b admin admin] PATH_INFO: `/v3/projects/48b41c830eb2403f8b3067e220af3589` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:15.455965 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdf47ef-e7b0-4260-9ba7-0f06aa65d61b admin admin] RBAC: Authorizing `identity:delete_project(project_id=48b41c830eb2403f8b3067e220af3589)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:15.458117 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdf47ef-e7b0-4260-9ba7-0f06aa65d61b admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:15.510400 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-3d92724c-79c2-4d93-b131-2619bec8a673 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:15.517477 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:15.518569 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-3d92724c-79c2-4d93-b131-2619bec8a673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:15.568403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 844/1698] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:15.576384 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ca2d1-0244-49bc-8fa5-b9dcb0e2f311 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:15.578850 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ca2d1-0244-49bc-8fa5-b9dcb0e2f311 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:15.582325 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ca2d1-0244-49bc-8fa5-b9dcb0e2f311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:15.628533 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-5fdf47ef-e7b0-4260-9ba7-0f06aa65d61b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48b41c830eb2403f8b3067e220af3589', 'request_id': 'req-5fdf47ef-e7b0-4260-9ba7-0f06aa65d61b'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:15.645094 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-5fdf47ef-e7b0-4260-9ba7-0f06aa65d61b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48b41c830eb2403f8b3067e220af3589', 'request_id': 'req-5fdf47ef-e7b0-4260-9ba7-0f06aa65d61b'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:15.647449 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-882ca2d1-0244-49bc-8fa5-b9dcb0e2f311 None None] MFA Rules not processed for user `f3185bd2458c4e22830975700888e9be`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:15.653109 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-882ca2d1-0244-49bc-8fa5-b9dcb0e2f311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:15.663790 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-5fdf47ef-e7b0-4260-9ba7-0f06aa65d61b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48b41c830eb2403f8b3067e220af3589', 'request_id': 'req-5fdf47ef-e7b0-4260-9ba7-0f06aa65d61b'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:15.693697 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-5fdf47ef-e7b0-4260-9ba7-0f06aa65d61b admin admin] The token cache is being invalidate because project 48b41c830eb2403f8b3067e220af3589 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:15.693978 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-5fdf47ef-e7b0-4260-9ba7-0f06aa65d61b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:15.697922 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-5fdf47ef-e7b0-4260-9ba7-0f06aa65d61b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:15.709931 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 855/1699] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:14 2022] DELETE /identity/v3/projects/48b41c830eb2403f8b3067e220af3589 => generated 0 bytes in 725 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:15.711647 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9ee0d-84ce-442c-b88f-c03dd726195b None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:15.723027 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9ee0d-84ce-442c-b88f-c03dd726195b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:15.723027 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9ee0d-84ce-442c-b88f-c03dd726195b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:15.781115 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-31e9ee0d-84ce-442c-b88f-c03dd726195b None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:15.783784 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-31e9ee0d-84ce-442c-b88f-c03dd726195b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:15.939538 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 845/1700] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Jun 15 14:43:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:15.940683 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ca2d1-0244-49bc-8fa5-b9dcb0e2f311 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:15.944840 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-882ca2d1-0244-49bc-8fa5-b9dcb0e2f311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:15.971967 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 856/1701] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:15.972561 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1de9e-d23d-4e7c-ad81-fd69a3850698 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:15.972936 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1de9e-d23d-4e7c-ad81-fd69a3850698 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:15.973220 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1de9e-d23d-4e7c-ad81-fd69a3850698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:16.044636 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-c7f1de9e-d23d-4e7c-ad81-fd69a3850698 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:16.050482 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c7f1de9e-d23d-4e7c-ad81-fd69a3850698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:16.068976 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:16.075259 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 857/1702] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:16.088394 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1de9e-d23d-4e7c-ad81-fd69a3850698 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:16.108533 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ca2d1-0244-49bc-8fa5-b9dcb0e2f311 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:16.116430 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1de9e-d23d-4e7c-ad81-fd69a3850698 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:16.118713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e149a-381d-4944-a964-3b33ea6c5a25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:16.119429 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e149a-381d-4944-a964-3b33ea6c5a25 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:16.131440 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e149a-381d-4944-a964-3b33ea6c5a25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:16.132360 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e149a-381d-4944-a964-3b33ea6c5a25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:16.137182 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-439e149a-381d-4944-a964-3b33ea6c5a25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:16.140743 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d27f81-3763-4f7c-bab9-36fa1cbf04af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:16.144420 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d27f81-3763-4f7c-bab9-36fa1cbf04af admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:16.148179 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d27f81-3763-4f7c-bab9-36fa1cbf04af admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:16.148718 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d27f81-3763-4f7c-bab9-36fa1cbf04af admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:16.149956 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d27f81-3763-4f7c-bab9-36fa1cbf04af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:43:16.156953 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d27f81-3763-4f7c-bab9-36fa1cbf04af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:16.170482 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d27f81-3763-4f7c-bab9-36fa1cbf04af admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:16.172734 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e149a-381d-4944-a964-3b33ea6c5a25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:16.174150 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e149a-381d-4944-a964-3b33ea6c5a25 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:16.217029 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 846/1703] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:16.220867 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781c54d-bc7c-49b5-9b18-6ad25ed7eab5 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:16.222517 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781c54d-bc7c-49b5-9b18-6ad25ed7eab5 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:16.224230 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781c54d-bc7c-49b5-9b18-6ad25ed7eab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:16.242333 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 858/1704] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:43:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:43:16.244501 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d27f81-3763-4f7c-bab9-36fa1cbf04af admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:16.290054 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-3781c54d-bc7c-49b5-9b18-6ad25ed7eab5 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:16.296313 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-3781c54d-bc7c-49b5-9b18-6ad25ed7eab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:16.315869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d27f81-3763-4f7c-bab9-36fa1cbf04af admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:16.324216 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 847/1705] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:16.328063 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3781c54d-bc7c-49b5-9b18-6ad25ed7eab5 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:16.339714 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf78cf4-1e16-401a-ab12-56b478d7ae0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:16.341095 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf78cf4-1e16-401a-ab12-56b478d7ae0b admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:16.342230 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf78cf4-1e16-401a-ab12-56b478d7ae0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:16.343499 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf78cf4-1e16-401a-ab12-56b478d7ae0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:16.345817 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf78cf4-1e16-401a-ab12-56b478d7ae0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:16.353192 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf78cf4-1e16-401a-ab12-56b478d7ae0b admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:16.359064 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3781c54d-bc7c-49b5-9b18-6ad25ed7eab5 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:16.397384 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aa0425-6e86-44f1-a521-0ccebed3530d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:16.398637 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa0425-6e86-44f1-a521-0ccebed3530d admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:16.399155 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa0425-6e86-44f1-a521-0ccebed3530d admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:16.399607 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa0425-6e86-44f1-a521-0ccebed3530d admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:16.400505 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aa0425-6e86-44f1-a521-0ccebed3530d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:43:16.403094 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aa0425-6e86-44f1-a521-0ccebed3530d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:16.410740 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aa0425-6e86-44f1-a521-0ccebed3530d admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:16.453962 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 848/1706] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:43:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:43:16.454946 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aa0425-6e86-44f1-a521-0ccebed3530d admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:16.493167 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 859/1707] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:16 2022] POST /identity/v3/projects => generated 405 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:16.496737 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aa0425-6e86-44f1-a521-0ccebed3530d admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:16.498515 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf78cf4-1e16-401a-ab12-56b478d7ae0b admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:16.712809 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67568b7e-7c4d-4c7a-b7b3-a77691e95de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:16.713779 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67568b7e-7c4d-4c7a-b7b3-a77691e95de1 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:16.714231 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67568b7e-7c4d-4c7a-b7b3-a77691e95de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:16.716880 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67568b7e-7c4d-4c7a-b7b3-a77691e95de1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:16.718131 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67568b7e-7c4d-4c7a-b7b3-a77691e95de1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:16.723782 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67568b7e-7c4d-4c7a-b7b3-a77691e95de1 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:16.730328 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf78cf4-1e16-401a-ab12-56b478d7ae0b admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:16.741195 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d7f63-04df-436f-acd6-f2700f8a6154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:16.747974 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d7f63-04df-436f-acd6-f2700f8a6154 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:16.747974 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d7f63-04df-436f-acd6-f2700f8a6154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:16.747974 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d7f63-04df-436f-acd6-f2700f8a6154 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:16.747974 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86d7f63-04df-436f-acd6-f2700f8a6154 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:16.749260 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86d7f63-04df-436f-acd6-f2700f8a6154 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:16.767608 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:16.802948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 849/1708] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:16 2022] POST /identity/v3/projects => generated 377 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:16.810333 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67568b7e-7c4d-4c7a-b7b3-a77691e95de1 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:16.821549 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 860/1709] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:16 2022] POST /identity/v3/users => generated 303 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:16.826642 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d7f63-04df-436f-acd6-f2700f8a6154 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:16.852674 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d7f63-04df-436f-acd6-f2700f8a6154 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:16.977256 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67568b7e-7c4d-4c7a-b7b3-a77691e95de1 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:16.993773 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ca66b-d0d8-4af5-b72a-fb20532f6032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:16.996691 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ca66b-d0d8-4af5-b72a-fb20532f6032 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:16.998526 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ca66b-d0d8-4af5-b72a-fb20532f6032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:16.999148 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ca66b-d0d8-4af5-b72a-fb20532f6032 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:17.004500 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6ca66b-d0d8-4af5-b72a-fb20532f6032 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:17.012395 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6ca66b-d0d8-4af5-b72a-fb20532f6032 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:17.034450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5311e-ffea-48ac-8387-f23f0b32ce22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:17.035681 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5311e-ffea-48ac-8387-f23f0b32ce22 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:17.036230 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5311e-ffea-48ac-8387-f23f0b32ce22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:17.038190 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5311e-ffea-48ac-8387-f23f0b32ce22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:17.050539 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f5311e-ffea-48ac-8387-f23f0b32ce22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:17.054439 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f5311e-ffea-48ac-8387-f23f0b32ce22 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:17.066355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 850/1710] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:16 2022] POST /identity/v3/users => generated 289 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:17.068279 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ca66b-d0d8-4af5-b72a-fb20532f6032 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:17.084910 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 861/1711] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:16 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:17.092341 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5311e-ffea-48ac-8387-f23f0b32ce22 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:17.098501 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ca66b-d0d8-4af5-b72a-fb20532f6032 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:17.114469 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43556ce0-f053-4635-b1ed-e01f07659115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:17.115491 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43556ce0-f053-4635-b1ed-e01f07659115 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:17.115848 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43556ce0-f053-4635-b1ed-e01f07659115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:17.116205 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43556ce0-f053-4635-b1ed-e01f07659115 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:17.117047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43556ce0-f053-4635-b1ed-e01f07659115 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:17.118519 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5311e-ffea-48ac-8387-f23f0b32ce22 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:17.120291 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43556ce0-f053-4635-b1ed-e01f07659115 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:17.134376 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4262d5-1a8e-4607-8242-a89e255cbda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:17.135364 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4262d5-1a8e-4607-8242-a89e255cbda5 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:17.135820 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4262d5-1a8e-4607-8242-a89e255cbda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:17.136242 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4262d5-1a8e-4607-8242-a89e255cbda5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:17.138564 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 851/1712] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:17 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:17.138957 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4262d5-1a8e-4607-8242-a89e255cbda5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:17.142572 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4262d5-1a8e-4607-8242-a89e255cbda5 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:17.143814 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43556ce0-f053-4635-b1ed-e01f07659115 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:17.168510 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43556ce0-f053-4635-b1ed-e01f07659115 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:17.169115 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 862/1713] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:17 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:17.174015 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4262d5-1a8e-4607-8242-a89e255cbda5 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:17.189603 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e15e57-5dcb-4fbe-a7e5-509ba64a0f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:17.189907 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4262d5-1a8e-4607-8242-a89e255cbda5 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:17.190176 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e15e57-5dcb-4fbe-a7e5-509ba64a0f22 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:17.190520 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e15e57-5dcb-4fbe-a7e5-509ba64a0f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:17.190952 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e15e57-5dcb-4fbe-a7e5-509ba64a0f22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:17.192017 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e15e57-5dcb-4fbe-a7e5-509ba64a0f22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:17.195372 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e15e57-5dcb-4fbe-a7e5-509ba64a0f22 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:17.205128 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab9d10-d302-4cfb-950d-8ac846948acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:17.207200 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab9d10-d302-4cfb-950d-8ac846948acd admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:17.207877 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab9d10-d302-4cfb-950d-8ac846948acd admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:17.208348 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab9d10-d302-4cfb-950d-8ac846948acd admin admin] PATH_INFO: `/v3/projects/ec9f56a2d76948ad9aac8b6c193a1b03/users/cfa1f5e2269242cdb8dff0930764596b/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:17.231233 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 852/1714] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:17 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:17.232264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e15e57-5dcb-4fbe-a7e5-509ba64a0f22 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:17.233295 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab9d10-d302-4cfb-950d-8ac846948acd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec9f56a2d76948ad9aac8b6c193a1b03, user_id=cfa1f5e2269242cdb8dff0930764596b, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:17.237287 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab9d10-d302-4cfb-950d-8ac846948acd admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:17.249530 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e15e57-5dcb-4fbe-a7e5-509ba64a0f22 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:17.259541 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 863/1715] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:17 2022] PUT /identity/v3/projects/ec9f56a2d76948ad9aac8b6c193a1b03/users/cfa1f5e2269242cdb8dff0930764596b/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:17.265520 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15343a-fac0-46f3-a8b1-8e5811f16d3a None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:17.265969 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15343a-fac0-46f3-a8b1-8e5811f16d3a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:17.266578 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15343a-fac0-46f3-a8b1-8e5811f16d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:17.267515 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:17.297948 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-fc15343a-fac0-46f3-a8b1-8e5811f16d3a None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:17.299987 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-fc15343a-fac0-46f3-a8b1-8e5811f16d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:17.382522 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f2d20-3494-456e-badd-decbd63859e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:17.383747 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f2d20-3494-456e-badd-decbd63859e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:17.384226 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f2d20-3494-456e-badd-decbd63859e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:17.384523 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f2d20-3494-456e-badd-decbd63859e2 admin admin] PATH_INFO: `/v3/projects/66e62fec988b4c57bbe08c8c8d147251/users/0aa2a7e2db5c415fbac95ee89dfc990a/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:17.413734 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30f2d20-3494-456e-badd-decbd63859e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66e62fec988b4c57bbe08c8c8d147251, user_id=0aa2a7e2db5c415fbac95ee89dfc990a, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:17.417487 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30f2d20-3494-456e-badd-decbd63859e2 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:17.453724 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 864/1716] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:17.460266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4335a-3121-4417-92c1-cf9950cacfe3 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:17.462305 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4335a-3121-4417-92c1-cf9950cacfe3 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:17.465325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4335a-3121-4417-92c1-cf9950cacfe3 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:17.477030 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 865/1717] 10.222.0.45 () {54 vars in 921 bytes} [Wed Jun 15 14:43:17 2022] GET /identity => generated 270 bytes in 22 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:43:17.481398 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a4335a-3121-4417-92c1-cf9950cacfe3 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:17.499588 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 853/1718] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:17 2022] PUT /identity/v3/projects/66e62fec988b4c57bbe08c8c8d147251/users/0aa2a7e2db5c415fbac95ee89dfc990a/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:17.504276 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-77a4335a-3121-4417-92c1-cf9950cacfe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:17.506117 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855405f-b86c-4adc-907b-70630b98cd9d None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:17.506611 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855405f-b86c-4adc-907b-70630b98cd9d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:17.508449 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855405f-b86c-4adc-907b-70630b98cd9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:17.596935 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-f855405f-b86c-4adc-907b-70630b98cd9d None None] MFA Rules not processed for user `0aa2a7e2db5c415fbac95ee89dfc990a`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:17.599085 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f855405f-b86c-4adc-907b-70630b98cd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:17.687925 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a4335a-3121-4417-92c1-cf9950cacfe3 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:17.707652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeff5a6-f7fd-4d47-87f9-c72087e9a400 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:17.709326 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeff5a6-f7fd-4d47-87f9-c72087e9a400 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:17.712197 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeff5a6-f7fd-4d47-87f9-c72087e9a400 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:17.712625 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeff5a6-f7fd-4d47-87f9-c72087e9a400 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:17.731046 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:17.798033 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 854/1719] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:17.800154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf019956-147e-427c-ac77-03d6198d78f9 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:17.801629 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf019956-147e-427c-ac77-03d6198d78f9 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:17.803990 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf019956-147e-427c-ac77-03d6198d78f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:17.904623 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-cf019956-147e-427c-ac77-03d6198d78f9 None None] MFA Rules not processed for user `f3185bd2458c4e22830975700888e9be`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:17.906155 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-cf019956-147e-427c-ac77-03d6198d78f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:17.967588 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeff5a6-f7fd-4d47-87f9-c72087e9a400 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:17.969425 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeff5a6-f7fd-4d47-87f9-c72087e9a400 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:18.010478 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 866/1720] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:18.011690 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeff5a6-f7fd-4d47-87f9-c72087e9a400 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:18.051411 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeff5a6-f7fd-4d47-87f9-c72087e9a400 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:18.064243 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb71db-d6a1-4dd7-b048-d4059ecca773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:18.069213 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb71db-d6a1-4dd7-b048-d4059ecca773 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:18.069697 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb71db-d6a1-4dd7-b048-d4059ecca773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:18.070298 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb71db-d6a1-4dd7-b048-d4059ecca773 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:18.071594 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cb71db-d6a1-4dd7-b048-d4059ecca773 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:18.079880 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cb71db-d6a1-4dd7-b048-d4059ecca773 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:18.144980 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 867/1721] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:18 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:18.146633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb71db-d6a1-4dd7-b048-d4059ecca773 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:18.150473 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-39cb71db-d6a1-4dd7-b048-d4059ecca773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:18.171145 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 855/1722] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Jun 15 14:43:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:18.174637 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf019956-147e-427c-ac77-03d6198d78f9 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:18.209512 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf019956-147e-427c-ac77-03d6198d78f9 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:18.252904 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:18.394346 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb71db-d6a1-4dd7-b048-d4059ecca773 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:18.421222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5323d4-766e-4fb5-ac3e-cbf4cd5d9ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:18.422550 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5323d4-766e-4fb5-ac3e-cbf4cd5d9ad2 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:18.422958 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd91293-14d7-40a1-bb3d-ac5007540f28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:18.423173 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5323d4-766e-4fb5-ac3e-cbf4cd5d9ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:18.423656 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5323d4-766e-4fb5-ac3e-cbf4cd5d9ad2 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:18.424346 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd91293-14d7-40a1-bb3d-ac5007540f28 service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:18.424782 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd91293-14d7-40a1-bb3d-ac5007540f28 service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:18.425159 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5323d4-766e-4fb5-ac3e-cbf4cd5d9ad2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:18.425705 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd91293-14d7-40a1-bb3d-ac5007540f28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:18.429855 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5323d4-766e-4fb5-ac3e-cbf4cd5d9ad2 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:18.465323 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd91293-14d7-40a1-bb3d-ac5007540f28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:18.472631 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd91293-14d7-40a1-bb3d-ac5007540f28 service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:18.489298 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 856/1723] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:18 2022] POST /identity/v3/users => generated 287 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:18.492971 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5323d4-766e-4fb5-ac3e-cbf4cd5d9ad2 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:18.516176 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 868/1724] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:18.518151 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5323d4-766e-4fb5-ac3e-cbf4cd5d9ad2 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:18.539969 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570768f1-195c-4105-a597-ad4682541e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:18.542311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570768f1-195c-4105-a597-ad4682541e57 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:18.542889 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570768f1-195c-4105-a597-ad4682541e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:18.543513 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570768f1-195c-4105-a597-ad4682541e57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:18.544639 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570768f1-195c-4105-a597-ad4682541e57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:18.547958 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570768f1-195c-4105-a597-ad4682541e57 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:18.604439 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 857/1725] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:43:18.605754 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd91293-14d7-40a1-bb3d-ac5007540f28 service nova] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:18.646818 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd91293-14d7-40a1-bb3d-ac5007540f28 service nova] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:18.657283 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2215eb79-30b2-477b-8a3b-40d2f6e339f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:18.658317 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215eb79-30b2-477b-8a3b-40d2f6e339f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:18.658747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215eb79-30b2-477b-8a3b-40d2f6e339f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:18.659252 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215eb79-30b2-477b-8a3b-40d2f6e339f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:18.660149 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2215eb79-30b2-477b-8a3b-40d2f6e339f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:18.662700 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2215eb79-30b2-477b-8a3b-40d2f6e339f8 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:18.692176 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 869/1726] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:43:18.702077 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570768f1-195c-4105-a597-ad4682541e57 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:18.730601 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570768f1-195c-4105-a597-ad4682541e57 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:18.748170 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97aaf11-c8eb-4428-9c62-9ccea7428355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:18.749209 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97aaf11-c8eb-4428-9c62-9ccea7428355 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:18.749606 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97aaf11-c8eb-4428-9c62-9ccea7428355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:18.750064 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97aaf11-c8eb-4428-9c62-9ccea7428355 admin admin] PATH_INFO: `/v3/projects/76c458300a214825b96a73c4677d5bcf/users/badc2f54fb354ecd85466f55c67cc4c2/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:18.785719 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97aaf11-c8eb-4428-9c62-9ccea7428355 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76c458300a214825b96a73c4677d5bcf, user_id=badc2f54fb354ecd85466f55c67cc4c2, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:18.788944 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97aaf11-c8eb-4428-9c62-9ccea7428355 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:18.821037 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 858/1727] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:18 2022] PUT /identity/v3/projects/76c458300a214825b96a73c4677d5bcf/users/badc2f54fb354ecd85466f55c67cc4c2/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:18.835897 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58775f65-7103-4437-bb11-b237f950f374 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:18.836184 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58775f65-7103-4437-bb11-b237f950f374 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:18.836468 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58775f65-7103-4437-bb11-b237f950f374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:18.951688 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-58775f65-7103-4437-bb11-b237f950f374 None None] MFA Rules not processed for user `badc2f54fb354ecd85466f55c67cc4c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:18.964067 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-58775f65-7103-4437-bb11-b237f950f374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:19.110451 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:19.163977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 870/1728] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:19.169833 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97aaf11-c8eb-4428-9c62-9ccea7428355 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:19.439280 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97aaf11-c8eb-4428-9c62-9ccea7428355 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:19.465487 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925633ab-5291-4d7f-a2f9-ce54132228d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:19.467192 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925633ab-5291-4d7f-a2f9-ce54132228d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:19.467920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925633ab-5291-4d7f-a2f9-ce54132228d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:19.468625 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925633ab-5291-4d7f-a2f9-ce54132228d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:19.470043 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925633ab-5291-4d7f-a2f9-ce54132228d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:19.478952 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925633ab-5291-4d7f-a2f9-ce54132228d4 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:19.554158 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 859/1729] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:19 2022] POST /identity/v3/projects => generated 377 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:19.559409 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58775f65-7103-4437-bb11-b237f950f374 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:19.588680 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58775f65-7103-4437-bb11-b237f950f374 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:19.651425 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925633ab-5291-4d7f-a2f9-ce54132228d4 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:19.653827 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-925633ab-5291-4d7f-a2f9-ce54132228d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:19.779333 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf59e2-fadb-417f-8d5d-0da55f8d5e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:19.780028 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf59e2-fadb-417f-8d5d-0da55f8d5e03 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:19.780312 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf59e2-fadb-417f-8d5d-0da55f8d5e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:19.780590 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf59e2-fadb-417f-8d5d-0da55f8d5e03 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:19.781300 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf59e2-fadb-417f-8d5d-0da55f8d5e03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:19.783300 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf59e2-fadb-417f-8d5d-0da55f8d5e03 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:19.819460 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:19.837585 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 871/1730] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:19 2022] POST /identity/v3/users => generated 295 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:19.846458 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf59e2-fadb-417f-8d5d-0da55f8d5e03 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:19.866513 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf59e2-fadb-417f-8d5d-0da55f8d5e03 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:19.871733 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925633ab-5291-4d7f-a2f9-ce54132228d4 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:19.884250 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68145b64-40bd-437e-8356-024080447d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:19.884250 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68145b64-40bd-437e-8356-024080447d35 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:19.885107 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68145b64-40bd-437e-8356-024080447d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:19.885444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68145b64-40bd-437e-8356-024080447d35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:19.888485 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68145b64-40bd-437e-8356-024080447d35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:19.892253 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfdcbb1-08ad-4f39-a1e1-6a7f82613861 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:19.893348 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdcbb1-08ad-4f39-a1e1-6a7f82613861 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:19.893348 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdcbb1-08ad-4f39-a1e1-6a7f82613861 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:19.893912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdcbb1-08ad-4f39-a1e1-6a7f82613861 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:19.896662 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68145b64-40bd-437e-8356-024080447d35 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:19.919534 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfdcbb1-08ad-4f39-a1e1-6a7f82613861 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:19.921424 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfdcbb1-08ad-4f39-a1e1-6a7f82613861 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:19.928317 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 872/1731] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:19.929855 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68145b64-40bd-437e-8356-024080447d35 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:19.932570 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-68145b64-40bd-437e-8356-024080447d35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:19.963576 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 860/1732] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:43:19.965801 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfdcbb1-08ad-4f39-a1e1-6a7f82613861 service nova] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:19.982509 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfdcbb1-08ad-4f39-a1e1-6a7f82613861 service nova] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:19.983617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68145b64-40bd-437e-8356-024080447d35 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:19.988686 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ea49f-2ffb-46a9-8a11-3d2c96dba310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:19.990501 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ea49f-2ffb-46a9-8a11-3d2c96dba310 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:19.991018 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ea49f-2ffb-46a9-8a11-3d2c96dba310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:19.991411 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ea49f-2ffb-46a9-8a11-3d2c96dba310 admin admin] PATH_INFO: `/v3/projects/49cd9de2cbef47a1b4ead8d74b0af9cc/users/eaac7f0978264ba18e7b552eb42c5882/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:19.995179 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf22e60-b74a-439d-8ab4-461016079976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:19.996093 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf22e60-b74a-439d-8ab4-461016079976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:19.996382 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf22e60-b74a-439d-8ab4-461016079976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:19.996668 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf22e60-b74a-439d-8ab4-461016079976 admin admin] PATH_INFO: `/v3/projects/4d36485f72ad45ccbe7bf55f13fbf59d` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:19.999833 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf22e60-b74a-439d-8ab4-461016079976 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d36485f72ad45ccbe7bf55f13fbf59d)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:20.002193 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf22e60-b74a-439d-8ab4-461016079976 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:20.014470 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ea49f-2ffb-46a9-8a11-3d2c96dba310 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49cd9de2cbef47a1b4ead8d74b0af9cc, user_id=eaac7f0978264ba18e7b552eb42c5882, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:20.017815 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ea49f-2ffb-46a9-8a11-3d2c96dba310 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:20.053335 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 861/1733] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:19 2022] PUT /identity/v3/projects/49cd9de2cbef47a1b4ead8d74b0af9cc/users/eaac7f0978264ba18e7b552eb42c5882/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:20.059890 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ea49f-2ffb-46a9-8a11-3d2c96dba310 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:20.124373 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-bdf22e60-b74a-439d-8ab4-461016079976 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d36485f72ad45ccbe7bf55f13fbf59d', 'request_id': 'req-bdf22e60-b74a-439d-8ab4-461016079976'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:20.144381 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-bdf22e60-b74a-439d-8ab4-461016079976 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d36485f72ad45ccbe7bf55f13fbf59d', 'request_id': 'req-bdf22e60-b74a-439d-8ab4-461016079976'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:20.156324 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-bdf22e60-b74a-439d-8ab4-461016079976 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d36485f72ad45ccbe7bf55f13fbf59d', 'request_id': 'req-bdf22e60-b74a-439d-8ab4-461016079976'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:20.175916 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-bdf22e60-b74a-439d-8ab4-461016079976 admin admin] The token cache is being invalidate because project 4d36485f72ad45ccbe7bf55f13fbf59d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:20.176263 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-bdf22e60-b74a-439d-8ab4-461016079976 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:20.177557 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-bdf22e60-b74a-439d-8ab4-461016079976 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:20.184228 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 873/1734] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:19 2022] DELETE /identity/v3/projects/4d36485f72ad45ccbe7bf55f13fbf59d => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:20.185919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e456d-5db3-467f-acc7-94a27883deb8 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:20.186621 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e456d-5db3-467f-acc7-94a27883deb8 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:20.186915 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e456d-5db3-467f-acc7-94a27883deb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:20.212090 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-2f4e456d-5db3-467f-acc7-94a27883deb8 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:20.213465 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-2f4e456d-5db3-467f-acc7-94a27883deb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:20.230047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ea49f-2ffb-46a9-8a11-3d2c96dba310 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:20.282154 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:20.312917 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 874/1735] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:20.336423 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d288f-90c5-41f0-9f58-6bbd78e71ccd None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:20.336796 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d288f-90c5-41f0-9f58-6bbd78e71ccd None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:20.337148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d288f-90c5-41f0-9f58-6bbd78e71ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:20.359624 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4f954-4a2d-459e-abf8-2046cccf5192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:20.359624 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4f954-4a2d-459e-abf8-2046cccf5192 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:20.359624 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4f954-4a2d-459e-abf8-2046cccf5192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:20.360707 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4f954-4a2d-459e-abf8-2046cccf5192 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:20.361905 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f4f954-4a2d-459e-abf8-2046cccf5192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:20.366984 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f4f954-4a2d-459e-abf8-2046cccf5192 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:20.377096 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-e45d288f-90c5-41f0-9f58-6bbd78e71ccd None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:20.378505 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e45d288f-90c5-41f0-9f58-6bbd78e71ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:20.393125 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 862/1736] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:20.393588 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 875/1737] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:20.397465 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d288f-90c5-41f0-9f58-6bbd78e71ccd None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:20.397689 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4f954-4a2d-459e-abf8-2046cccf5192 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:20.398844 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e45d288f-90c5-41f0-9f58-6bbd78e71ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:20.413758 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d288f-90c5-41f0-9f58-6bbd78e71ccd None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:20.422713 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b949fa-64dc-4640-b6f5-de8795001d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:20.423213 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4f954-4a2d-459e-abf8-2046cccf5192 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:20.424327 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b949fa-64dc-4640-b6f5-de8795001d54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:20.426143 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b949fa-64dc-4640-b6f5-de8795001d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:20.426614 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b949fa-64dc-4640-b6f5-de8795001d54 admin admin] PATH_INFO: `/v3/projects/49cd9de2cbef47a1b4ead8d74b0af9cc/users/eaac7f0978264ba18e7b552eb42c5882/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:20.432084 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b949fa-64dc-4640-b6f5-de8795001d54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49cd9de2cbef47a1b4ead8d74b0af9cc, user_id=eaac7f0978264ba18e7b552eb42c5882, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:20.438639 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b949fa-64dc-4640-b6f5-de8795001d54 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:20.440743 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6ccaec-7201-4474-9349-9d0c42e3fa7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:20.441548 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ccaec-7201-4474-9349-9d0c42e3fa7a admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:20.441838 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ccaec-7201-4474-9349-9d0c42e3fa7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:20.442163 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ccaec-7201-4474-9349-9d0c42e3fa7a admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:20.443895 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6ccaec-7201-4474-9349-9d0c42e3fa7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:43:20.444403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6ccaec-7201-4474-9349-9d0c42e3fa7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:20.446407 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6ccaec-7201-4474-9349-9d0c42e3fa7a admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:20.478825 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 876/1738] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:20 2022] PUT /identity/v3/projects/49cd9de2cbef47a1b4ead8d74b0af9cc/users/eaac7f0978264ba18e7b552eb42c5882/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:20.479718 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 863/1739] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:43:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:43:20.490481 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b949fa-64dc-4640-b6f5-de8795001d54 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:20.492036 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254a17b-1d63-46bd-a6cd-deb7530dfc3b None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:20.492698 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254a17b-1d63-46bd-a6cd-deb7530dfc3b None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:20.493065 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254a17b-1d63-46bd-a6cd-deb7530dfc3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:20.546320 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-d254a17b-1d63-46bd-a6cd-deb7530dfc3b None None] MFA Rules not processed for user `eaac7f0978264ba18e7b552eb42c5882`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:20.547861 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d254a17b-1d63-46bd-a6cd-deb7530dfc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:20.634762 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:20.658919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b949fa-64dc-4640-b6f5-de8795001d54 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:20.661995 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8c1c4-619b-470a-a27d-f606c3973a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:20.662890 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8c1c4-619b-470a-a27d-f606c3973a6d admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:20.663325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8c1c4-619b-470a-a27d-f606c3973a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:20.663734 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8c1c4-619b-470a-a27d-f606c3973a6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:20.664674 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e8c1c4-619b-470a-a27d-f606c3973a6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:20.666958 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e8c1c4-619b-470a-a27d-f606c3973a6d admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:20.691326 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 864/1740] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:20 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:20.714313 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 877/1741] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:20 2022] POST /identity/v3/projects => generated 379 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:20.714518 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d254a17b-1d63-46bd-a6cd-deb7530dfc3b None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:20.716878 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d254a17b-1d63-46bd-a6cd-deb7530dfc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:20.718261 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8c1c4-619b-470a-a27d-f606c3973a6d admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:20.853184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d254a17b-1d63-46bd-a6cd-deb7530dfc3b None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:20.915839 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8c1c4-619b-470a-a27d-f606c3973a6d admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:20.939230 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e2345-f2af-4679-8b2d-f5190ab0b5f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:20.939460 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f35d56-033f-48e9-b390-d71dd5c4fb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:20.939631 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e2345-f2af-4679-8b2d-f5190ab0b5f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:20.940002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e2345-f2af-4679-8b2d-f5190ab0b5f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:20.940388 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e2345-f2af-4679-8b2d-f5190ab0b5f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:20.941204 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f35d56-033f-48e9-b390-d71dd5c4fb8e admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:20.941769 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f35d56-033f-48e9-b390-d71dd5c4fb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:20.942328 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f35d56-033f-48e9-b390-d71dd5c4fb8e admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:20.943361 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f35d56-033f-48e9-b390-d71dd5c4fb8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:20.945538 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f35d56-033f-48e9-b390-d71dd5c4fb8e admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:20.988441 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 878/1742] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:20 2022] POST /identity/v3/users => generated 290 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:20.994717 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f35d56-033f-48e9-b390-d71dd5c4fb8e admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:21.015308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f35d56-033f-48e9-b390-d71dd5c4fb8e admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:21.032335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2815e3f-d152-443c-8e5d-ddbf01e3c426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:21.034733 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2815e3f-d152-443c-8e5d-ddbf01e3c426 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:21.035266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2815e3f-d152-443c-8e5d-ddbf01e3c426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:21.035710 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2815e3f-d152-443c-8e5d-ddbf01e3c426 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:21.036725 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2815e3f-d152-443c-8e5d-ddbf01e3c426 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:21.037305 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e2345-f2af-4679-8b2d-f5190ab0b5f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:21.038765 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2815e3f-d152-443c-8e5d-ddbf01e3c426 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:21.040118 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e2345-f2af-4679-8b2d-f5190ab0b5f0 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:21.053751 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 865/1743] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:20 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:21.061078 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 879/1744] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:21.066475 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e2345-f2af-4679-8b2d-f5190ab0b5f0 service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:21.081812 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e2345-f2af-4679-8b2d-f5190ab0b5f0 service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:21.087694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127869c7-5870-4ac0-9aed-ce3ccb1a49b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:21.088520 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127869c7-5870-4ac0-9aed-ce3ccb1a49b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:21.088716 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127869c7-5870-4ac0-9aed-ce3ccb1a49b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:21.088993 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127869c7-5870-4ac0-9aed-ce3ccb1a49b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:21.089760 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127869c7-5870-4ac0-9aed-ce3ccb1a49b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:21.108700 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127869c7-5870-4ac0-9aed-ce3ccb1a49b8 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:21.109792 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2815e3f-d152-443c-8e5d-ddbf01e3c426 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:21.113838 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a2815e3f-d152-443c-8e5d-ddbf01e3c426 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:21.128875 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 866/1745] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:21.129820 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127869c7-5870-4ac0-9aed-ce3ccb1a49b8 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:21.136566 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2815e3f-d152-443c-8e5d-ddbf01e3c426 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:21.141204 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127869c7-5870-4ac0-9aed-ce3ccb1a49b8 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:21.146800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d30fe22-b97e-4e5c-a18a-5873e4c2618b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:21.147794 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d30fe22-b97e-4e5c-a18a-5873e4c2618b admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:21.148075 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d30fe22-b97e-4e5c-a18a-5873e4c2618b admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:21.148350 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d30fe22-b97e-4e5c-a18a-5873e4c2618b admin admin] PATH_INFO: `/v3/projects/40a7c2e314ac4335b18f08359e7f66f2/users/f53604bd286c4676a51e77582910de8f/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:21.148582 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643ab18-4c32-4b83-8389-c4c723463bb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:21.151094 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643ab18-4c32-4b83-8389-c4c723463bb5 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:21.151821 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643ab18-4c32-4b83-8389-c4c723463bb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:21.152379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643ab18-4c32-4b83-8389-c4c723463bb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:21.155099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e643ab18-4c32-4b83-8389-c4c723463bb5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:21.173776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d30fe22-b97e-4e5c-a18a-5873e4c2618b admin admin] RBAC: Authorizing `identity:create_grant(project_id=40a7c2e314ac4335b18f08359e7f66f2, user_id=f53604bd286c4676a51e77582910de8f, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:21.177697 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d30fe22-b97e-4e5c-a18a-5873e4c2618b admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:21.204895 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 867/1746] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:21 2022] PUT /identity/v3/projects/40a7c2e314ac4335b18f08359e7f66f2/users/f53604bd286c4676a51e77582910de8f/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:21.215366 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d3db5-9781-4205-9a23-12fd3c055774 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:21.215690 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d3db5-9781-4205-9a23-12fd3c055774 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:21.215983 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d3db5-9781-4205-9a23-12fd3c055774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:21.216125 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:21.225919 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:21.273136 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e643ab18-4c32-4b83-8389-c4c723463bb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:21.274607 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e643ab18-4c32-4b83-8389-c4c723463bb5 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:21.287131 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-295d3db5-9781-4205-9a23-12fd3c055774 None None] MFA Rules not processed for user `f53604bd286c4676a51e77582910de8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:21.288606 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-295d3db5-9781-4205-9a23-12fd3c055774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:21.290974 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 880/1747] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:21.472591 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 868/1748] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:21.478192 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643ab18-4c32-4b83-8389-c4c723463bb5 service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:21.497566 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643ab18-4c32-4b83-8389-c4c723463bb5 service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:21.544877 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1bcba-9d56-4a3e-ab4d-f15610c36414 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:21.545180 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1bcba-9d56-4a3e-ab4d-f15610c36414 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:21.545462 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1bcba-9d56-4a3e-ab4d-f15610c36414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:21.617374 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-18d1bcba-9d56-4a3e-ab4d-f15610c36414 None None] MFA Rules not processed for user `cfa1f5e2269242cdb8dff0930764596b`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:21.620648 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-18d1bcba-9d56-4a3e-ab4d-f15610c36414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:21.670688 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dba5c9-b226-4041-acf7-850a4c5fcc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:21.671834 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dba5c9-b226-4041-acf7-850a4c5fcc9a admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:21.673520 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dba5c9-b226-4041-acf7-850a4c5fcc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:21.673907 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dba5c9-b226-4041-acf7-850a4c5fcc9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:21.676022 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dba5c9-b226-4041-acf7-850a4c5fcc9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:21.686099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dba5c9-b226-4041-acf7-850a4c5fcc9a admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:21.779687 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 881/1749] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:21 2022] POST /identity/v3/projects => generated 379 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:21.782813 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dba5c9-b226-4041-acf7-850a4c5fcc9a admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:21.884824 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 869/1750] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:21 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:43:21.887968 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d1bcba-9d56-4a3e-ab4d-f15610c36414 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:21.890038 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-18d1bcba-9d56-4a3e-ab4d-f15610c36414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:21.965229 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d1bcba-9d56-4a3e-ab4d-f15610c36414 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:22.047998 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dba5c9-b226-4041-acf7-850a4c5fcc9a admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:22.072847 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c0220-bd3c-4fa3-9fc6-fa42622f7088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:22.076740 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c0220-bd3c-4fa3-9fc6-fa42622f7088 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:22.077263 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c0220-bd3c-4fa3-9fc6-fa42622f7088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:22.079158 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c0220-bd3c-4fa3-9fc6-fa42622f7088 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:22.083031 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c0220-bd3c-4fa3-9fc6-fa42622f7088 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:22.093282 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c0220-bd3c-4fa3-9fc6-fa42622f7088 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:22.153140 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 882/1751] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:21 2022] POST /identity/v3/users => generated 296 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:22.158363 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c0220-bd3c-4fa3-9fc6-fa42622f7088 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:22.190790 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c0220-bd3c-4fa3-9fc6-fa42622f7088 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:22.205616 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979637b3-7076-490e-ab34-775e6c0c696b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:22.207852 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979637b3-7076-490e-ab34-775e6c0c696b admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:22.208382 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979637b3-7076-490e-ab34-775e6c0c696b admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:22.211012 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979637b3-7076-490e-ab34-775e6c0c696b admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:22.211800 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979637b3-7076-490e-ab34-775e6c0c696b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:22.216486 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979637b3-7076-490e-ab34-775e6c0c696b admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:22.225412 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0442636-3bc7-4b37-982b-526fe1bb1a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:22.226144 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0442636-3bc7-4b37-982b-526fe1bb1a39 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:22.226435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0442636-3bc7-4b37-982b-526fe1bb1a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:22.226712 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0442636-3bc7-4b37-982b-526fe1bb1a39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:22.227446 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0442636-3bc7-4b37-982b-526fe1bb1a39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:22.229410 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0442636-3bc7-4b37-982b-526fe1bb1a39 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:22.252551 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 883/1752] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:22.256974 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979637b3-7076-490e-ab34-775e6c0c696b admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:22.269648 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979637b3-7076-490e-ab34-775e6c0c696b admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:22.275547 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1743ca51-8177-4762-9a87-e5137fb38123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:22.277143 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743ca51-8177-4762-9a87-e5137fb38123 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:22.277428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743ca51-8177-4762-9a87-e5137fb38123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:22.277701 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743ca51-8177-4762-9a87-e5137fb38123 admin admin] PATH_INFO: `/v3/projects/dff4c772dc2d42209a7771e2e1cc848c/users/4637bd688ad74a31a739759076a523d3/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:22.297383 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 870/1753] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:21 2022] POST /identity/v3/projects => generated 405 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:22.300454 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0442636-3bc7-4b37-982b-526fe1bb1a39 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:22.304179 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1743ca51-8177-4762-9a87-e5137fb38123 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dff4c772dc2d42209a7771e2e1cc848c, user_id=4637bd688ad74a31a739759076a523d3, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:22.316367 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:22.316456 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1743ca51-8177-4762-9a87-e5137fb38123 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:22.348877 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 884/1754] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:22 2022] PUT /identity/v3/projects/dff4c772dc2d42209a7771e2e1cc848c/users/4637bd688ad74a31a739759076a523d3/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:43:22.352517 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1743ca51-8177-4762-9a87-e5137fb38123 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:22.376155 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:22.384434 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0442636-3bc7-4b37-982b-526fe1bb1a39 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:22.460221 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe05dc-1bf4-4546-bbe2-85688e1e0349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:22.461815 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe05dc-1bf4-4546-bbe2-85688e1e0349 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:22.462292 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe05dc-1bf4-4546-bbe2-85688e1e0349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:22.462761 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe05dc-1bf4-4546-bbe2-85688e1e0349 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:22.464238 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbe05dc-1bf4-4546-bbe2-85688e1e0349 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:22.466531 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbe05dc-1bf4-4546-bbe2-85688e1e0349 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:22.491564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1743ca51-8177-4762-9a87-e5137fb38123 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:22.495369 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 871/1755] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:22 2022] POST /identity/v3/users => generated 309 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:43:22.500098 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe05dc-1bf4-4546-bbe2-85688e1e0349 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:22.503669 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad357c6-b41d-459f-b201-04f957b80b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:22.504437 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad357c6-b41d-459f-b201-04f957b80b9b admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:22.504929 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad357c6-b41d-459f-b201-04f957b80b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:22.505816 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad357c6-b41d-459f-b201-04f957b80b9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:22.506694 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad357c6-b41d-459f-b201-04f957b80b9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:22.510949 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad357c6-b41d-459f-b201-04f957b80b9b admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:22.513216 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe05dc-1bf4-4546-bbe2-85688e1e0349 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:22.522031 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c447e-bc4e-418d-8fa4-df6492fd3a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:22.523127 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c447e-bc4e-418d-8fa4-df6492fd3a5f admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:22.523563 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c447e-bc4e-418d-8fa4-df6492fd3a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:22.524053 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c447e-bc4e-418d-8fa4-df6492fd3a5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:22.524954 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c447e-bc4e-418d-8fa4-df6492fd3a5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:22.527279 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c447e-bc4e-418d-8fa4-df6492fd3a5f admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:22.537828 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 885/1756] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:43:22.539030 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad357c6-b41d-459f-b201-04f957b80b9b admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:22.549263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 872/1757] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:22.550273 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c447e-bc4e-418d-8fa4-df6492fd3a5f admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:22.563722 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad357c6-b41d-459f-b201-04f957b80b9b admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:22.566427 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c447e-bc4e-418d-8fa4-df6492fd3a5f admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:22.576680 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f88517-d08e-4873-a763-c58b6153010b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:22.576930 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea29af6-ec3f-447a-9594-97a9fd947de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:22.577664 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f88517-d08e-4873-a763-c58b6153010b admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:22.577948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f88517-d08e-4873-a763-c58b6153010b admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:22.578223 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f88517-d08e-4873-a763-c58b6153010b admin admin] PATH_INFO: `/v3/projects/76c4218ecf254b02874caba3e45afbc2/users/371729b403f5463faa484d3a21bdfc6a/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:22.579427 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea29af6-ec3f-447a-9594-97a9fd947de8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:22.580508 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea29af6-ec3f-447a-9594-97a9fd947de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:22.580858 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea29af6-ec3f-447a-9594-97a9fd947de8 admin admin] PATH_INFO: `/v3/projects/dff4c772dc2d42209a7771e2e1cc848c/users/4637bd688ad74a31a739759076a523d3/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:22.586054 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea29af6-ec3f-447a-9594-97a9fd947de8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dff4c772dc2d42209a7771e2e1cc848c, user_id=4637bd688ad74a31a739759076a523d3, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:22.592638 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea29af6-ec3f-447a-9594-97a9fd947de8 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:22.624653 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f88517-d08e-4873-a763-c58b6153010b admin admin] RBAC: Authorizing `identity:create_grant(project_id=76c4218ecf254b02874caba3e45afbc2, user_id=371729b403f5463faa484d3a21bdfc6a, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:22.624927 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 886/1758] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:22 2022] PUT /identity/v3/projects/dff4c772dc2d42209a7771e2e1cc848c/users/4637bd688ad74a31a739759076a523d3/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:22.637743 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29c1ed-28a0-4c27-b0bf-a81bf44e08c3 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:22.638099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29c1ed-28a0-4c27-b0bf-a81bf44e08c3 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:22.638347 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29c1ed-28a0-4c27-b0bf-a81bf44e08c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:22.639074 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f88517-d08e-4873-a763-c58b6153010b admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:22.656479 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 873/1759] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:22 2022] PUT /identity/v3/projects/76c4218ecf254b02874caba3e45afbc2/users/371729b403f5463faa484d3a21bdfc6a/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:22.660973 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f88517-d08e-4873-a763-c58b6153010b admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:22.683538 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-7e29c1ed-28a0-4c27-b0bf-a81bf44e08c3 None None] MFA Rules not processed for user `4637bd688ad74a31a739759076a523d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:22.685956 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-7e29c1ed-28a0-4c27-b0bf-a81bf44e08c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:22.779692 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f88517-d08e-4873-a763-c58b6153010b admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:22.787753 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e7492-cba6-4cd9-9308-b091b674a0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:22.792600 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e7492-cba6-4cd9-9308-b091b674a0b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:22.793140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e7492-cba6-4cd9-9308-b091b674a0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:22.793622 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e7492-cba6-4cd9-9308-b091b674a0b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:22.794564 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e7492-cba6-4cd9-9308-b091b674a0b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:22.797285 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e7492-cba6-4cd9-9308-b091b674a0b8 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:22.814924 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 874/1760] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:22.815920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e7492-cba6-4cd9-9308-b091b674a0b8 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:22.828923 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e7492-cba6-4cd9-9308-b091b674a0b8 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:22.834518 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93e8581-9f7c-425e-86be-41e172d3353c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:22.835777 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e8581-9f7c-425e-86be-41e172d3353c admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:22.836142 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e8581-9f7c-425e-86be-41e172d3353c admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:22.836572 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e8581-9f7c-425e-86be-41e172d3353c admin admin] PATH_INFO: `/v3/projects/76c4218ecf254b02874caba3e45afbc2/users/371729b403f5463faa484d3a21bdfc6a/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:22.839409 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93e8581-9f7c-425e-86be-41e172d3353c admin admin] RBAC: Authorizing `identity:create_grant(project_id=76c4218ecf254b02874caba3e45afbc2, user_id=371729b403f5463faa484d3a21bdfc6a, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:22.842097 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93e8581-9f7c-425e-86be-41e172d3353c admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:22.846771 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:22.872644 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 875/1761] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:22 2022] PUT /identity/v3/projects/76c4218ecf254b02874caba3e45afbc2/users/371729b403f5463faa484d3a21bdfc6a/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:22.935562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 887/1762] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:22 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:22.938187 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93e8581-9f7c-425e-86be-41e172d3353c admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:23.056284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93e8581-9f7c-425e-86be-41e172d3353c admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:23.061084 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a2e4bd-6509-44d2-925e-de06d5264cd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:23.061868 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a2e4bd-6509-44d2-925e-de06d5264cd5 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:23.062188 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a2e4bd-6509-44d2-925e-de06d5264cd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:23.062550 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a2e4bd-6509-44d2-925e-de06d5264cd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:23.203245 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a2e4bd-6509-44d2-925e-de06d5264cd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:23.204775 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a2e4bd-6509-44d2-925e-de06d5264cd5 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:23.224363 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 876/1763] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:22 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:43:23.353340 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e29c1ed-28a0-4c27-b0bf-a81bf44e08c3 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:23.444295 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e29c1ed-28a0-4c27-b0bf-a81bf44e08c3 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:23.450221 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc70c88-d34a-41c7-b7a0-651be3e2fe87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:23.451011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc70c88-d34a-41c7-b7a0-651be3e2fe87 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:23.451372 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc70c88-d34a-41c7-b7a0-651be3e2fe87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:23.451838 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc70c88-d34a-41c7-b7a0-651be3e2fe87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:23.498197 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:23.555485 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc70c88-d34a-41c7-b7a0-651be3e2fe87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:23.557403 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc70c88-d34a-41c7-b7a0-651be3e2fe87 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:23.573206 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 888/1764] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:24.143996 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4066ec-64a9-49d8-a803-7a5c0f1930a0 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:24.144521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4066ec-64a9-49d8-a803-7a5c0f1930a0 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:24.144521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4066ec-64a9-49d8-a803-7a5c0f1930a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:24.325858 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-fa4066ec-64a9-49d8-a803-7a5c0f1930a0 None None] MFA Rules not processed for user `371729b403f5463faa484d3a21bdfc6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:24.328373 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-fa4066ec-64a9-49d8-a803-7a5c0f1930a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:24.435634 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 877/1765] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:24 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:25.253421 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb6bc4-113f-42d8-964b-2c6b2ebe6455 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:25.254137 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb6bc4-113f-42d8-964b-2c6b2ebe6455 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:25.254421 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb6bc4-113f-42d8-964b-2c6b2ebe6455 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:25.258087 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 889/1766] 10.222.0.45 () {54 vars in 921 bytes} [Wed Jun 15 14:43:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:43:25.266125 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa571a7-25ef-408a-b178-d384c1940254 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:25.266585 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa571a7-25ef-408a-b178-d384c1940254 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:25.266893 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa571a7-25ef-408a-b178-d384c1940254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:25.304726 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-6fa571a7-25ef-408a-b178-d384c1940254 None None] MFA Rules not processed for user `f4128111a9aa450abb2146a5d9926dec`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:25.306127 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-6fa571a7-25ef-408a-b178-d384c1940254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:25.356116 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:25.404293 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 878/1767] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Jun 15 14:43:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:43:25.417225 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb6bc4-113f-42d8-964b-2c6b2ebe6455 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:25.419571 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-19cb6bc4-113f-42d8-964b-2c6b2ebe6455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:25.446413 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb6bc4-113f-42d8-964b-2c6b2ebe6455 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:25.452632 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff142af1-8636-44eb-85f6-4572b59b6195 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4128111a9aa450abb2146a5d9926dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:25.453476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff142af1-8636-44eb-85f6-4572b59b6195 service placement] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:25.453913 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff142af1-8636-44eb-85f6-4572b59b6195 service placement] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:25.454350 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff142af1-8636-44eb-85f6-4572b59b6195 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:25.467606 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff142af1-8636-44eb-85f6-4572b59b6195 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:25.470863 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff142af1-8636-44eb-85f6-4572b59b6195 service placement] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:25.481685 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 890/1768] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:26.025598 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeef61d-716e-4c4d-8d1a-da51bad1adef None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:26.026226 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeef61d-716e-4c4d-8d1a-da51bad1adef None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:26.026514 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeef61d-716e-4c4d-8d1a-da51bad1adef None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:26.027941 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 879/1769] 10.222.0.45 () {54 vars in 921 bytes} [Wed Jun 15 14:43:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:43:26.043480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed3c3f-16eb-4557-92bc-24e39331b0e6 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:26.043778 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed3c3f-16eb-4557-92bc-24e39331b0e6 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:26.044063 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed3c3f-16eb-4557-92bc-24e39331b0e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:26.062856 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-98ed3c3f-16eb-4557-92bc-24e39331b0e6 None None] MFA Rules not processed for user `f4128111a9aa450abb2146a5d9926dec`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:26.064123 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-98ed3c3f-16eb-4557-92bc-24e39331b0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:26.073231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 891/1770] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Jun 15 14:43:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:43:26.095350 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeef61d-716e-4c4d-8d1a-da51bad1adef None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:26.106538 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed3c3f-16eb-4557-92bc-24e39331b0e6 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:26.112134 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeef61d-716e-4c4d-8d1a-da51bad1adef None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:26.119385 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a43668-90fc-4c1b-b0f4-196e5ce0c6a7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4128111a9aa450abb2146a5d9926dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:26.120103 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a43668-90fc-4c1b-b0f4-196e5ce0c6a7 service placement] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:26.120476 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a43668-90fc-4c1b-b0f4-196e5ce0c6a7 service placement] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:26.122222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a43668-90fc-4c1b-b0f4-196e5ce0c6a7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:26.130226 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a43668-90fc-4c1b-b0f4-196e5ce0c6a7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:26.131944 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a43668-90fc-4c1b-b0f4-196e5ce0c6a7 service placement] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:26.141934 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 880/1771] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:26.180461 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed3c3f-16eb-4557-92bc-24e39331b0e6 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:26.185349 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c0237-0349-482f-ad9d-ea74cee6e5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:26.186234 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c0237-0349-482f-ad9d-ea74cee6e5b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:26.186541 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c0237-0349-482f-ad9d-ea74cee6e5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:26.186820 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c0237-0349-482f-ad9d-ea74cee6e5b4 admin admin] PATH_INFO: `/v3/users/cfa1f5e2269242cdb8dff0930764596b` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:26.188083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0c0237-0349-482f-ad9d-ea74cee6e5b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfa1f5e2269242cdb8dff0930764596b)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:26.190040 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0c0237-0349-482f-ad9d-ea74cee6e5b4 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:26.243011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-0c0c0237-0349-482f-ad9d-ea74cee6e5b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfa1f5e2269242cdb8dff0930764596b'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:26.243720 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-0c0c0237-0349-482f-ad9d-ea74cee6e5b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfa1f5e2269242cdb8dff0930764596b'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:26.263529 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-0c0c0237-0349-482f-ad9d-ea74cee6e5b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfa1f5e2269242cdb8dff0930764596b'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:26.264440 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-0c0c0237-0349-482f-ad9d-ea74cee6e5b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfa1f5e2269242cdb8dff0930764596b'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:26.272269 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-0c0c0237-0349-482f-ad9d-ea74cee6e5b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfa1f5e2269242cdb8dff0930764596b'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:26.287755 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 892/1772] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:26 2022] DELETE /identity/v3/users/cfa1f5e2269242cdb8dff0930764596b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:43:26.563560 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a43668-90fc-4c1b-b0f4-196e5ce0c6a7 service placement] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:26.577374 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a43668-90fc-4c1b-b0f4-196e5ce0c6a7 service placement] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:26.657217 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eca6e0-4d0f-4a9c-91cb-667f2aaf8757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:26.658322 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eca6e0-4d0f-4a9c-91cb-667f2aaf8757 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:26.658615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eca6e0-4d0f-4a9c-91cb-667f2aaf8757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:26.658911 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eca6e0-4d0f-4a9c-91cb-667f2aaf8757 admin admin] PATH_INFO: `/v3/projects/ec9f56a2d76948ad9aac8b6c193a1b03` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:26.660282 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eca6e0-4d0f-4a9c-91cb-667f2aaf8757 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec9f56a2d76948ad9aac8b6c193a1b03)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:26.662681 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eca6e0-4d0f-4a9c-91cb-667f2aaf8757 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:26.726808 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-68eca6e0-4d0f-4a9c-91cb-667f2aaf8757 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec9f56a2d76948ad9aac8b6c193a1b03', 'request_id': 'req-68eca6e0-4d0f-4a9c-91cb-667f2aaf8757'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:26.738703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-68eca6e0-4d0f-4a9c-91cb-667f2aaf8757 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec9f56a2d76948ad9aac8b6c193a1b03', 'request_id': 'req-68eca6e0-4d0f-4a9c-91cb-667f2aaf8757'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:26.746061 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-68eca6e0-4d0f-4a9c-91cb-667f2aaf8757 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec9f56a2d76948ad9aac8b6c193a1b03', 'request_id': 'req-68eca6e0-4d0f-4a9c-91cb-667f2aaf8757'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:26.756592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-68eca6e0-4d0f-4a9c-91cb-667f2aaf8757 admin admin] The token cache is being invalidate because project ec9f56a2d76948ad9aac8b6c193a1b03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:26.756857 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-68eca6e0-4d0f-4a9c-91cb-667f2aaf8757 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:26.757575 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-68eca6e0-4d0f-4a9c-91cb-667f2aaf8757 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:26.759156 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 881/1773] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:26 2022] DELETE /identity/v3/projects/ec9f56a2d76948ad9aac8b6c193a1b03 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:43:26.763232 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c0237-0349-482f-ad9d-ea74cee6e5b4 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:26.765206 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-0c0c0237-0349-482f-ad9d-ea74cee6e5b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:26.780506 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c0237-0349-482f-ad9d-ea74cee6e5b4 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:26.819514 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:26.855772 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17896565-ab5f-4554-a7c5-5a9903be9bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:26.857363 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17896565-ab5f-4554-a7c5-5a9903be9bf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:26.857715 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17896565-ab5f-4554-a7c5-5a9903be9bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:26.858041 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17896565-ab5f-4554-a7c5-5a9903be9bf2 admin admin] PATH_INFO: `/v3/users/371729b403f5463faa484d3a21bdfc6a` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:26.859510 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17896565-ab5f-4554-a7c5-5a9903be9bf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=371729b403f5463faa484d3a21bdfc6a)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:26.861901 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17896565-ab5f-4554-a7c5-5a9903be9bf2 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:26.917858 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-17896565-ab5f-4554-a7c5-5a9903be9bf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '371729b403f5463faa484d3a21bdfc6a'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:26.918444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-17896565-ab5f-4554-a7c5-5a9903be9bf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '371729b403f5463faa484d3a21bdfc6a'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:26.930725 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-17896565-ab5f-4554-a7c5-5a9903be9bf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '371729b403f5463faa484d3a21bdfc6a'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:26.931398 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-17896565-ab5f-4554-a7c5-5a9903be9bf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '371729b403f5463faa484d3a21bdfc6a'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:26.938714 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-17896565-ab5f-4554-a7c5-5a9903be9bf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '371729b403f5463faa484d3a21bdfc6a'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:26.951702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 893/1774] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:26 2022] DELETE /identity/v3/users/371729b403f5463faa484d3a21bdfc6a => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:27.126127 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eca6e0-4d0f-4a9c-91cb-667f2aaf8757 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:27.127781 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-68eca6e0-4d0f-4a9c-91cb-667f2aaf8757 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:27.141116 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eca6e0-4d0f-4a9c-91cb-667f2aaf8757 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:27.190637 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:27.208548 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a14041-0d8d-45e1-b5ae-d0f84e9fad86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:27.209414 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a14041-0d8d-45e1-b5ae-d0f84e9fad86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:27.209694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a14041-0d8d-45e1-b5ae-d0f84e9fad86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:27.209967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a14041-0d8d-45e1-b5ae-d0f84e9fad86 admin admin] PATH_INFO: `/v3/projects/76c4218ecf254b02874caba3e45afbc2` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:27.211173 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a14041-0d8d-45e1-b5ae-d0f84e9fad86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76c4218ecf254b02874caba3e45afbc2)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:27.213087 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a14041-0d8d-45e1-b5ae-d0f84e9fad86 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:27.259994 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-88a14041-0d8d-45e1-b5ae-d0f84e9fad86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76c4218ecf254b02874caba3e45afbc2', 'request_id': 'req-88a14041-0d8d-45e1-b5ae-d0f84e9fad86'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:27.267400 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-88a14041-0d8d-45e1-b5ae-d0f84e9fad86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76c4218ecf254b02874caba3e45afbc2', 'request_id': 'req-88a14041-0d8d-45e1-b5ae-d0f84e9fad86'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:27.273709 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-88a14041-0d8d-45e1-b5ae-d0f84e9fad86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76c4218ecf254b02874caba3e45afbc2', 'request_id': 'req-88a14041-0d8d-45e1-b5ae-d0f84e9fad86'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:27.284175 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-88a14041-0d8d-45e1-b5ae-d0f84e9fad86 admin admin] The token cache is being invalidate because project 76c4218ecf254b02874caba3e45afbc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:27.284466 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-88a14041-0d8d-45e1-b5ae-d0f84e9fad86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:27.285103 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-88a14041-0d8d-45e1-b5ae-d0f84e9fad86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:27.286721 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 882/1775] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:27 2022] DELETE /identity/v3/projects/76c4218ecf254b02874caba3e45afbc2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:27.291716 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ba5f0-6626-4900-921a-a74820948f89 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:27.292182 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ba5f0-6626-4900-921a-a74820948f89 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:27.292298 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ba5f0-6626-4900-921a-a74820948f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:27.318347 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-b51ba5f0-6626-4900-921a-a74820948f89 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:27.319796 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-b51ba5f0-6626-4900-921a-a74820948f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:27.378198 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 894/1776] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:27.387944 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8bf14-98ec-4198-acb9-08965b273fd5 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:27.388229 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8bf14-98ec-4198-acb9-08965b273fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:27.388499 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8bf14-98ec-4198-acb9-08965b273fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:27.407940 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-70c8bf14-98ec-4198-acb9-08965b273fd5 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:27.409138 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-70c8bf14-98ec-4198-acb9-08965b273fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:27.417308 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 883/1777] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:27.421969 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ba5f0-6626-4900-921a-a74820948f89 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:27.433946 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ba5f0-6626-4900-921a-a74820948f89 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:27.438472 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0100e-6773-431b-91dc-aa07d0f05eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:27.439179 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0100e-6773-431b-91dc-aa07d0f05eb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:27.439463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0100e-6773-431b-91dc-aa07d0f05eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:27.439736 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0100e-6773-431b-91dc-aa07d0f05eb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:27.440461 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0100e-6773-431b-91dc-aa07d0f05eb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:43:27.440817 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0100e-6773-431b-91dc-aa07d0f05eb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:27.442361 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0100e-6773-431b-91dc-aa07d0f05eb0 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:27.454559 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 895/1778] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:43:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:43:27.459539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8bf14-98ec-4198-acb9-08965b273fd5 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:27.469240 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8bf14-98ec-4198-acb9-08965b273fd5 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:27.473916 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799068e1-f995-4f62-80b7-e320ff77d866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:27.475185 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799068e1-f995-4f62-80b7-e320ff77d866 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:27.475471 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799068e1-f995-4f62-80b7-e320ff77d866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:27.475750 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799068e1-f995-4f62-80b7-e320ff77d866 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:27.476438 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799068e1-f995-4f62-80b7-e320ff77d866 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:27.478340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799068e1-f995-4f62-80b7-e320ff77d866 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:27.503953 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 884/1779] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:27 2022] POST /identity/v3/projects => generated 363 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:27.508336 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0100e-6773-431b-91dc-aa07d0f05eb0 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:27.518385 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0100e-6773-431b-91dc-aa07d0f05eb0 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:27.552183 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:27.581771 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9f988-24ce-4bc1-a38d-4263bb8f22e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:27.582452 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9f988-24ce-4bc1-a38d-4263bb8f22e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:27.582740 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9f988-24ce-4bc1-a38d-4263bb8f22e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:27.583050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9f988-24ce-4bc1-a38d-4263bb8f22e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:27.584407 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e9f988-24ce-4bc1-a38d-4263bb8f22e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:27.586353 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e9f988-24ce-4bc1-a38d-4263bb8f22e6 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:27.606974 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 896/1780] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:27 2022] POST /identity/v3/users => generated 282 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:27.611475 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799068e1-f995-4f62-80b7-e320ff77d866 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:27.622570 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799068e1-f995-4f62-80b7-e320ff77d866 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:27.627624 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a979b22e-2c4e-4b9e-9f61-cc2988fd9ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:27.628294 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979b22e-2c4e-4b9e-9f61-cc2988fd9ef3 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:27.628581 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979b22e-2c4e-4b9e-9f61-cc2988fd9ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:27.628858 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979b22e-2c4e-4b9e-9f61-cc2988fd9ef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:27.629644 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a979b22e-2c4e-4b9e-9f61-cc2988fd9ef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:27.631884 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a979b22e-2c4e-4b9e-9f61-cc2988fd9ef3 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:27.644079 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 885/1781] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:27.648982 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9f988-24ce-4bc1-a38d-4263bb8f22e6 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:27.661042 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9f988-24ce-4bc1-a38d-4263bb8f22e6 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:27.667472 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da81614-1263-433e-82ad-b8c6067016c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:27.668229 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da81614-1263-433e-82ad-b8c6067016c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:27.668572 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da81614-1263-433e-82ad-b8c6067016c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:27.668871 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da81614-1263-433e-82ad-b8c6067016c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:27.669579 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da81614-1263-433e-82ad-b8c6067016c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:27.671284 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da81614-1263-433e-82ad-b8c6067016c0 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:27.683698 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 897/1782] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:43:27.688655 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a979b22e-2c4e-4b9e-9f61-cc2988fd9ef3 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:27.705017 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a979b22e-2c4e-4b9e-9f61-cc2988fd9ef3 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:27.720838 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df52448-2a9e-489a-bc75-3cc155f2bb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:27.722814 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df52448-2a9e-489a-bc75-3cc155f2bb5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:27.722814 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df52448-2a9e-489a-bc75-3cc155f2bb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:27.722814 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df52448-2a9e-489a-bc75-3cc155f2bb5d admin admin] PATH_INFO: `/v3/projects/f0033a6ad1eb49b3b5beea5f35f56953/users/3397f10f22df41fd9b54ced10a21e625/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:27.743547 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df52448-2a9e-489a-bc75-3cc155f2bb5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0033a6ad1eb49b3b5beea5f35f56953, user_id=3397f10f22df41fd9b54ced10a21e625, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:27.746440 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df52448-2a9e-489a-bc75-3cc155f2bb5d admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:27.761754 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 886/1783] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:27 2022] PUT /identity/v3/projects/f0033a6ad1eb49b3b5beea5f35f56953/users/3397f10f22df41fd9b54ced10a21e625/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:27.773081 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b77397-5400-473f-afa9-5990cc8df5be None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:27.773393 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b77397-5400-473f-afa9-5990cc8df5be None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:27.773661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b77397-5400-473f-afa9-5990cc8df5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:27.806746 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-c5b77397-5400-473f-afa9-5990cc8df5be None None] MFA Rules not processed for user `3397f10f22df41fd9b54ced10a21e625`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:27.808857 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c5b77397-5400-473f-afa9-5990cc8df5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:27.901399 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 898/1784] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:27 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:43:27.906759 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df52448-2a9e-489a-bc75-3cc155f2bb5d admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:28.013744 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df52448-2a9e-489a-bc75-3cc155f2bb5d admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:28.030003 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da7426-18e4-459d-a167-43f72bc1f476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:28.031158 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da7426-18e4-459d-a167-43f72bc1f476 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:28.031617 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da7426-18e4-459d-a167-43f72bc1f476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:28.032087 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da7426-18e4-459d-a167-43f72bc1f476 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:28.033297 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28da7426-18e4-459d-a167-43f72bc1f476 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:28.042520 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28da7426-18e4-459d-a167-43f72bc1f476 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:28.058062 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:28.087083 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 887/1785] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:27 2022] POST /identity/v3/projects => generated 365 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:28.088277 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b77397-5400-473f-afa9-5990cc8df5be None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:28.103351 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b77397-5400-473f-afa9-5990cc8df5be None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:28.184692 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da7426-18e4-459d-a167-43f72bc1f476 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:28.186958 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-28da7426-18e4-459d-a167-43f72bc1f476 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:28.201447 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d2c3e-1960-4bbd-832f-348a2e6eb226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:28.203450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d2c3e-1960-4bbd-832f-348a2e6eb226 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:28.204082 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d2c3e-1960-4bbd-832f-348a2e6eb226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:28.204611 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d2c3e-1960-4bbd-832f-348a2e6eb226 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:28.206015 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d2c3e-1960-4bbd-832f-348a2e6eb226 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:28.208170 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d2c3e-1960-4bbd-832f-348a2e6eb226 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:28.233960 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 899/1786] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:28 2022] POST /identity/v3/users => generated 289 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:28.241288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d2c3e-1960-4bbd-832f-348a2e6eb226 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:28.261274 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d2c3e-1960-4bbd-832f-348a2e6eb226 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:28.266600 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb7454-c150-46b6-8da9-42118173d225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:28.267578 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb7454-c150-46b6-8da9-42118173d225 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:28.268514 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb7454-c150-46b6-8da9-42118173d225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:28.268680 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb7454-c150-46b6-8da9-42118173d225 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:28.269841 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb7454-c150-46b6-8da9-42118173d225 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:28.271872 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb7454-c150-46b6-8da9-42118173d225 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:28.287362 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 900/1787] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:28 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:28.288614 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb7454-c150-46b6-8da9-42118173d225 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:28.293543 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da7426-18e4-459d-a167-43f72bc1f476 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:28.302792 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827d305-335c-4d3e-8d5f-cfa0156b161f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:28.303760 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827d305-335c-4d3e-8d5f-cfa0156b161f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:28.304254 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827d305-335c-4d3e-8d5f-cfa0156b161f admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:28.304399 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827d305-335c-4d3e-8d5f-cfa0156b161f admin admin] PATH_INFO: `/v3/users/0aa2a7e2db5c415fbac95ee89dfc990a` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:28.305688 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d827d305-335c-4d3e-8d5f-cfa0156b161f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0aa2a7e2db5c415fbac95ee89dfc990a)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:28.305803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb7454-c150-46b6-8da9-42118173d225 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:28.307787 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d827d305-335c-4d3e-8d5f-cfa0156b161f admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:28.311303 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4111c01-efb1-425b-ae43-64dd55eaba0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:28.312499 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4111c01-efb1-425b-ae43-64dd55eaba0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:28.313129 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4111c01-efb1-425b-ae43-64dd55eaba0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:28.313370 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4111c01-efb1-425b-ae43-64dd55eaba0d admin admin] PATH_INFO: `/v3/projects/a9b798fa233243d19a7910ff98075abf/users/afe85fbb5b6d4b32a0a0e990762b67b1/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:28.340223 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4111c01-efb1-425b-ae43-64dd55eaba0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9b798fa233243d19a7910ff98075abf, user_id=afe85fbb5b6d4b32a0a0e990762b67b1, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:28.345199 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4111c01-efb1-425b-ae43-64dd55eaba0d admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:28.358200 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:28.370071 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 901/1788] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:28 2022] PUT /identity/v3/projects/a9b798fa233243d19a7910ff98075abf/users/afe85fbb5b6d4b32a0a0e990762b67b1/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:28.371338 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4111c01-efb1-425b-ae43-64dd55eaba0d admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:28.393151 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-d827d305-335c-4d3e-8d5f-cfa0156b161f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0aa2a7e2db5c415fbac95ee89dfc990a'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:28.394730 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-d827d305-335c-4d3e-8d5f-cfa0156b161f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0aa2a7e2db5c415fbac95ee89dfc990a'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:28.420854 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-d827d305-335c-4d3e-8d5f-cfa0156b161f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0aa2a7e2db5c415fbac95ee89dfc990a'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:28.429653 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-d827d305-335c-4d3e-8d5f-cfa0156b161f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0aa2a7e2db5c415fbac95ee89dfc990a'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:28.437926 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-d827d305-335c-4d3e-8d5f-cfa0156b161f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0aa2a7e2db5c415fbac95ee89dfc990a'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:28.445335 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 888/1789] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:28 2022] DELETE /identity/v3/users/0aa2a7e2db5c415fbac95ee89dfc990a => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:43:28.449851 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b818bf3-9655-4407-bef4-5f45ae04b0b5 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:28.450147 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b818bf3-9655-4407-bef4-5f45ae04b0b5 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:28.450434 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b818bf3-9655-4407-bef4-5f45ae04b0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:28.467687 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4111c01-efb1-425b-ae43-64dd55eaba0d admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:28.468713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-0b818bf3-9655-4407-bef4-5f45ae04b0b5 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:28.469924 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-0b818bf3-9655-4407-bef4-5f45ae04b0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:28.527574 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f21cf09-54f7-482b-a180-9b77de200d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:28.528255 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21cf09-54f7-482b-a180-9b77de200d0f admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:28.528540 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21cf09-54f7-482b-a180-9b77de200d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:28.528822 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21cf09-54f7-482b-a180-9b77de200d0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:28.529545 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f21cf09-54f7-482b-a180-9b77de200d0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:28.531173 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f21cf09-54f7-482b-a180-9b77de200d0f admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:28.537013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 889/1790] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:43:28.541240 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 902/1791] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:28 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:28.545194 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b818bf3-9655-4407-bef4-5f45ae04b0b5 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:28.546785 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-0b818bf3-9655-4407-bef4-5f45ae04b0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:28.547109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f21cf09-54f7-482b-a180-9b77de200d0f admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:28.548727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-8f21cf09-54f7-482b-a180-9b77de200d0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:28.561351 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b818bf3-9655-4407-bef4-5f45ae04b0b5 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:28.566837 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb4a2c5-f652-43ea-b6f9-88e4a2a8a11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:28.568308 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb4a2c5-f652-43ea-b6f9-88e4a2a8a11b admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:28.568590 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb4a2c5-f652-43ea-b6f9-88e4a2a8a11b admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:28.568867 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb4a2c5-f652-43ea-b6f9-88e4a2a8a11b admin admin] PATH_INFO: `/v3/projects/a9b798fa233243d19a7910ff98075abf/users/afe85fbb5b6d4b32a0a0e990762b67b1/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:28.570851 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb4a2c5-f652-43ea-b6f9-88e4a2a8a11b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9b798fa233243d19a7910ff98075abf, user_id=afe85fbb5b6d4b32a0a0e990762b67b1, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:28.573528 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb4a2c5-f652-43ea-b6f9-88e4a2a8a11b admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:28.580438 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:28.589554 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 890/1792] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:28 2022] PUT /identity/v3/projects/a9b798fa233243d19a7910ff98075abf/users/afe85fbb5b6d4b32a0a0e990762b67b1/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:43:28.597882 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109e723-dde0-4b6c-b1cb-410461de7e6c None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:28.598179 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109e723-dde0-4b6c-b1cb-410461de7e6c None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:28.598472 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109e723-dde0-4b6c-b1cb-410461de7e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:28.613977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f21cf09-54f7-482b-a180-9b77de200d0f admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:28.625617 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-d109e723-dde0-4b6c-b1cb-410461de7e6c None None] MFA Rules not processed for user `afe85fbb5b6d4b32a0a0e990762b67b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:28.626874 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d109e723-dde0-4b6c-b1cb-410461de7e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:28.665373 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747b143-746a-4ea5-906e-834b8f90660f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:28.666046 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747b143-746a-4ea5-906e-834b8f90660f service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:28.666334 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747b143-746a-4ea5-906e-834b8f90660f service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:28.666617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747b143-746a-4ea5-906e-834b8f90660f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:28.680490 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:28.711957 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 891/1793] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:28 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:43:28.728366 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d109e723-dde0-4b6c-b1cb-410461de7e6c None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:28.728366 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d109e723-dde0-4b6c-b1cb-410461de7e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:28.738387 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f747b143-746a-4ea5-906e-834b8f90660f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:28.740688 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f747b143-746a-4ea5-906e-834b8f90660f service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:28.751113 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 903/1794] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:43:28.794283 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d109e723-dde0-4b6c-b1cb-410461de7e6c None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:28.798573 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7239e635-e0ca-45dd-9cc0-5341092b13d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:28.799271 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239e635-e0ca-45dd-9cc0-5341092b13d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:28.799560 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239e635-e0ca-45dd-9cc0-5341092b13d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:28.799837 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239e635-e0ca-45dd-9cc0-5341092b13d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:28.809926 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7239e635-e0ca-45dd-9cc0-5341092b13d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:28.811450 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7239e635-e0ca-45dd-9cc0-5341092b13d1 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:28.830793 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 892/1795] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:28 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:43:29.330752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747b143-746a-4ea5-906e-834b8f90660f service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:29.332845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f747b143-746a-4ea5-906e-834b8f90660f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:29.350538 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7239e635-e0ca-45dd-9cc0-5341092b13d1 service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:29.364053 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7239e635-e0ca-45dd-9cc0-5341092b13d1 service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:29.372668 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747b143-746a-4ea5-906e-834b8f90660f service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:29.378298 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b04efc6-95b6-4844-87c6-f29d5750aa4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:29.379223 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04efc6-95b6-4844-87c6-f29d5750aa4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:29.379506 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04efc6-95b6-4844-87c6-f29d5750aa4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:29.379802 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04efc6-95b6-4844-87c6-f29d5750aa4b admin admin] PATH_INFO: `/v3/users/3397f10f22df41fd9b54ced10a21e625` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:29.381165 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2211856-2dc4-4cb4-9835-635a3cb647ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:29.381687 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b04efc6-95b6-4844-87c6-f29d5750aa4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3397f10f22df41fd9b54ced10a21e625)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:29.382518 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2211856-2dc4-4cb4-9835-635a3cb647ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:29.382811 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2211856-2dc4-4cb4-9835-635a3cb647ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:29.383123 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2211856-2dc4-4cb4-9835-635a3cb647ec admin admin] PATH_INFO: `/v3/projects/66e62fec988b4c57bbe08c8c8d147251` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:29.383898 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b04efc6-95b6-4844-87c6-f29d5750aa4b admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:29.384915 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2211856-2dc4-4cb4-9835-635a3cb647ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=66e62fec988b4c57bbe08c8c8d147251)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:29.386885 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2211856-2dc4-4cb4-9835-635a3cb647ec admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:29.448454 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-7b04efc6-95b6-4844-87c6-f29d5750aa4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3397f10f22df41fd9b54ced10a21e625'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:29.449224 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-7b04efc6-95b6-4844-87c6-f29d5750aa4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3397f10f22df41fd9b54ced10a21e625'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:29.450179 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e2211856-2dc4-4cb4-9835-635a3cb647ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66e62fec988b4c57bbe08c8c8d147251', 'request_id': 'req-e2211856-2dc4-4cb4-9835-635a3cb647ec'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:29.460569 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e2211856-2dc4-4cb4-9835-635a3cb647ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66e62fec988b4c57bbe08c8c8d147251', 'request_id': 'req-e2211856-2dc4-4cb4-9835-635a3cb647ec'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:29.465727 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-7b04efc6-95b6-4844-87c6-f29d5750aa4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3397f10f22df41fd9b54ced10a21e625'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:29.469807 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e2211856-2dc4-4cb4-9835-635a3cb647ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66e62fec988b4c57bbe08c8c8d147251', 'request_id': 'req-e2211856-2dc4-4cb4-9835-635a3cb647ec'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:29.477524 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-7b04efc6-95b6-4844-87c6-f29d5750aa4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3397f10f22df41fd9b54ced10a21e625'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:29.484155 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e2211856-2dc4-4cb4-9835-635a3cb647ec admin admin] The token cache is being invalidate because project 66e62fec988b4c57bbe08c8c8d147251 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:29.484596 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e2211856-2dc4-4cb4-9835-635a3cb647ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:29.486413 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e2211856-2dc4-4cb4-9835-635a3cb647ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:29.487931 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-7b04efc6-95b6-4844-87c6-f29d5750aa4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3397f10f22df41fd9b54ced10a21e625'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:29.490772 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:29.491115 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 904/1796] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:29 2022] DELETE /identity/v3/projects/66e62fec988b4c57bbe08c8c8d147251 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:29.497592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2211856-2dc4-4cb4-9835-635a3cb647ec admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:29.497844 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 893/1797] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:29 2022] DELETE /identity/v3/users/3397f10f22df41fd9b54ced10a21e625 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:43:29.501694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e061e38-a3a4-4e84-a408-3d36aa54937d None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:29.502252 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e061e38-a3a4-4e84-a408-3d36aa54937d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:29.502690 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e2211856-2dc4-4cb4-9835-635a3cb647ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:29.503241 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e061e38-a3a4-4e84-a408-3d36aa54937d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:29.525728 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-8e061e38-a3a4-4e84-a408-3d36aa54937d None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:29.527230 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-8e061e38-a3a4-4e84-a408-3d36aa54937d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:29.577549 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2211856-2dc4-4cb4-9835-635a3cb647ec admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:29.591336 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 894/1798] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:43:29.602014 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e061e38-a3a4-4e84-a408-3d36aa54937d None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:29.603940 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-8e061e38-a3a4-4e84-a408-3d36aa54937d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:29.620672 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:29.643620 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba6678a-d1a5-4948-8b1f-fd3e75a0c322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:29.644508 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6678a-d1a5-4948-8b1f-fd3e75a0c322 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:29.644794 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6678a-d1a5-4948-8b1f-fd3e75a0c322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:29.645114 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6678a-d1a5-4948-8b1f-fd3e75a0c322 admin admin] PATH_INFO: `/v3/users/badc2f54fb354ecd85466f55c67cc4c2` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:29.647345 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba6678a-d1a5-4948-8b1f-fd3e75a0c322 admin admin] RBAC: Authorizing `identity:delete_user(user_id=badc2f54fb354ecd85466f55c67cc4c2)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:29.649120 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba6678a-d1a5-4948-8b1f-fd3e75a0c322 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:29.665887 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e061e38-a3a4-4e84-a408-3d36aa54937d None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:29.669615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99568a38-13d2-4512-9efb-2713b8410a11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:29.670275 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99568a38-13d2-4512-9efb-2713b8410a11 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:29.670560 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99568a38-13d2-4512-9efb-2713b8410a11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:29.670839 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99568a38-13d2-4512-9efb-2713b8410a11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:29.683502 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99568a38-13d2-4512-9efb-2713b8410a11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:29.685448 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99568a38-13d2-4512-9efb-2713b8410a11 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:29.702203 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 895/1799] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:43:29.705647 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-0ba6678a-d1a5-4948-8b1f-fd3e75a0c322 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'badc2f54fb354ecd85466f55c67cc4c2'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:29.706339 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-0ba6678a-d1a5-4948-8b1f-fd3e75a0c322 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'badc2f54fb354ecd85466f55c67cc4c2'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:29.719366 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-0ba6678a-d1a5-4948-8b1f-fd3e75a0c322 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'badc2f54fb354ecd85466f55c67cc4c2'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:29.720131 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-0ba6678a-d1a5-4948-8b1f-fd3e75a0c322 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'badc2f54fb354ecd85466f55c67cc4c2'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:29.732237 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-0ba6678a-d1a5-4948-8b1f-fd3e75a0c322 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'badc2f54fb354ecd85466f55c67cc4c2'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:29.748428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 905/1800] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:29 2022] DELETE /identity/v3/users/badc2f54fb354ecd85466f55c67cc4c2 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:43:30.130609 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99568a38-13d2-4512-9efb-2713b8410a11 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:30.132497 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-99568a38-13d2-4512-9efb-2713b8410a11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:30.146950 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99568a38-13d2-4512-9efb-2713b8410a11 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:30.213181 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba6678a-d1a5-4948-8b1f-fd3e75a0c322 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:30.214567 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-0ba6678a-d1a5-4948-8b1f-fd3e75a0c322 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:30.225673 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30adb60-2106-401c-b0f5-95cbc2d5aa71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:30.226557 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30adb60-2106-401c-b0f5-95cbc2d5aa71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:30.226846 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30adb60-2106-401c-b0f5-95cbc2d5aa71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:30.227164 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30adb60-2106-401c-b0f5-95cbc2d5aa71 admin admin] PATH_INFO: `/v3/projects/f0033a6ad1eb49b3b5beea5f35f56953` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:30.228384 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30adb60-2106-401c-b0f5-95cbc2d5aa71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0033a6ad1eb49b3b5beea5f35f56953)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:30.230315 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30adb60-2106-401c-b0f5-95cbc2d5aa71 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:30.299042 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:30.308389 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e30adb60-2106-401c-b0f5-95cbc2d5aa71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0033a6ad1eb49b3b5beea5f35f56953', 'request_id': 'req-e30adb60-2106-401c-b0f5-95cbc2d5aa71'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:30.314932 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba6678a-d1a5-4948-8b1f-fd3e75a0c322 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:30.322415 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e30adb60-2106-401c-b0f5-95cbc2d5aa71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0033a6ad1eb49b3b5beea5f35f56953', 'request_id': 'req-e30adb60-2106-401c-b0f5-95cbc2d5aa71'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:30.335669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e30adb60-2106-401c-b0f5-95cbc2d5aa71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0033a6ad1eb49b3b5beea5f35f56953', 'request_id': 'req-e30adb60-2106-401c-b0f5-95cbc2d5aa71'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:30.352982 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e30adb60-2106-401c-b0f5-95cbc2d5aa71 admin admin] The token cache is being invalidate because project f0033a6ad1eb49b3b5beea5f35f56953 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:30.353378 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e30adb60-2106-401c-b0f5-95cbc2d5aa71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:30.354171 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e30adb60-2106-401c-b0f5-95cbc2d5aa71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:30.361280 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 896/1801] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:30 2022] DELETE /identity/v3/projects/f0033a6ad1eb49b3b5beea5f35f56953 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:30.362376 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30adb60-2106-401c-b0f5-95cbc2d5aa71 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:30.365307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e30adb60-2106-401c-b0f5-95cbc2d5aa71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:30.402079 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa555eb0-7995-4cbc-a553-d59b268e0585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:30.403189 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa555eb0-7995-4cbc-a553-d59b268e0585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:30.403543 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa555eb0-7995-4cbc-a553-d59b268e0585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:30.403816 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa555eb0-7995-4cbc-a553-d59b268e0585 admin admin] PATH_INFO: `/v3/projects/76c458300a214825b96a73c4677d5bcf` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:30.405142 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa555eb0-7995-4cbc-a553-d59b268e0585 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76c458300a214825b96a73c4677d5bcf)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:30.407138 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa555eb0-7995-4cbc-a553-d59b268e0585 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:30.416435 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:30.456740 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30adb60-2106-401c-b0f5-95cbc2d5aa71 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:30.474414 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-aa555eb0-7995-4cbc-a553-d59b268e0585 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76c458300a214825b96a73c4677d5bcf', 'request_id': 'req-aa555eb0-7995-4cbc-a553-d59b268e0585'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:30.487704 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-aa555eb0-7995-4cbc-a553-d59b268e0585 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76c458300a214825b96a73c4677d5bcf', 'request_id': 'req-aa555eb0-7995-4cbc-a553-d59b268e0585'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:30.496413 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-aa555eb0-7995-4cbc-a553-d59b268e0585 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76c458300a214825b96a73c4677d5bcf', 'request_id': 'req-aa555eb0-7995-4cbc-a553-d59b268e0585'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:30.547599 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-aa555eb0-7995-4cbc-a553-d59b268e0585 admin admin] The token cache is being invalidate because project 76c458300a214825b96a73c4677d5bcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:30.547899 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-aa555eb0-7995-4cbc-a553-d59b268e0585 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:30.548911 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-aa555eb0-7995-4cbc-a553-d59b268e0585 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:30.551929 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 906/1802] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:30 2022] DELETE /identity/v3/projects/76c458300a214825b96a73c4677d5bcf => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:43:30.553154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0604a7c-6079-409b-9de6-7052aa76afc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:30.554346 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0604a7c-6079-409b-9de6-7052aa76afc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:30.555207 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0604a7c-6079-409b-9de6-7052aa76afc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:30.555743 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0604a7c-6079-409b-9de6-7052aa76afc9 admin admin] PATH_INFO: `/v3/users/afe85fbb5b6d4b32a0a0e990762b67b1` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:30.558061 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0604a7c-6079-409b-9de6-7052aa76afc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afe85fbb5b6d4b32a0a0e990762b67b1)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:30.558208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa555eb0-7995-4cbc-a553-d59b268e0585 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:30.561532 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0604a7c-6079-409b-9de6-7052aa76afc9 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:30.564912 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-aa555eb0-7995-4cbc-a553-d59b268e0585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:30.580241 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa555eb0-7995-4cbc-a553-d59b268e0585 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:30.586709 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f414f56-d281-4e4e-b0ec-ab91974888f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:30.587668 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f414f56-d281-4e4e-b0ec-ab91974888f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:30.587950 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f414f56-d281-4e4e-b0ec-ab91974888f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:30.588282 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f414f56-d281-4e4e-b0ec-ab91974888f8 admin admin] PATH_INFO: `/v3/users/eaac7f0978264ba18e7b552eb42c5882` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:30.589688 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f414f56-d281-4e4e-b0ec-ab91974888f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaac7f0978264ba18e7b552eb42c5882)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:30.591919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f414f56-d281-4e4e-b0ec-ab91974888f8 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:30.666157 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b0604a7c-6079-409b-9de6-7052aa76afc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afe85fbb5b6d4b32a0a0e990762b67b1'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:30.666933 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b0604a7c-6079-409b-9de6-7052aa76afc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afe85fbb5b6d4b32a0a0e990762b67b1'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:30.669415 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-2f414f56-d281-4e4e-b0ec-ab91974888f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaac7f0978264ba18e7b552eb42c5882'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:30.670339 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-2f414f56-d281-4e4e-b0ec-ab91974888f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaac7f0978264ba18e7b552eb42c5882'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:30.685815 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b0604a7c-6079-409b-9de6-7052aa76afc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afe85fbb5b6d4b32a0a0e990762b67b1'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:30.690182 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-2f414f56-d281-4e4e-b0ec-ab91974888f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaac7f0978264ba18e7b552eb42c5882'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:30.691416 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-2f414f56-d281-4e4e-b0ec-ab91974888f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaac7f0978264ba18e7b552eb42c5882'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:30.696342 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b0604a7c-6079-409b-9de6-7052aa76afc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afe85fbb5b6d4b32a0a0e990762b67b1'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:30.711402 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-2f414f56-d281-4e4e-b0ec-ab91974888f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaac7f0978264ba18e7b552eb42c5882'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:30.716112 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b0604a7c-6079-409b-9de6-7052aa76afc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afe85fbb5b6d4b32a0a0e990762b67b1'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:30.718605 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:30.726017 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 897/1803] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:30 2022] DELETE /identity/v3/users/afe85fbb5b6d4b32a0a0e990762b67b1 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:30.734585 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 907/1804] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:30 2022] DELETE /identity/v3/users/eaac7f0978264ba18e7b552eb42c5882 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:31.041283 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0604a7c-6079-409b-9de6-7052aa76afc9 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:31.043027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b0604a7c-6079-409b-9de6-7052aa76afc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:31.133171 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0604a7c-6079-409b-9de6-7052aa76afc9 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:31.196046 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7cb382-a0b5-4ebf-86ba-07e652515e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:31.196909 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cb382-a0b5-4ebf-86ba-07e652515e4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:31.197575 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cb382-a0b5-4ebf-86ba-07e652515e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:31.197575 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cb382-a0b5-4ebf-86ba-07e652515e4a admin admin] PATH_INFO: `/v3/projects/a9b798fa233243d19a7910ff98075abf` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:31.198649 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7cb382-a0b5-4ebf-86ba-07e652515e4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9b798fa233243d19a7910ff98075abf)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:31.200594 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7cb382-a0b5-4ebf-86ba-07e652515e4a admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:31.250129 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f414f56-d281-4e4e-b0ec-ab91974888f8 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:31.251936 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-2f414f56-d281-4e4e-b0ec-ab91974888f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:31.259508 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-8d7cb382-a0b5-4ebf-86ba-07e652515e4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9b798fa233243d19a7910ff98075abf', 'request_id': 'req-8d7cb382-a0b5-4ebf-86ba-07e652515e4a'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:31.268235 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f414f56-d281-4e4e-b0ec-ab91974888f8 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:31.269161 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-8d7cb382-a0b5-4ebf-86ba-07e652515e4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9b798fa233243d19a7910ff98075abf', 'request_id': 'req-8d7cb382-a0b5-4ebf-86ba-07e652515e4a'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:31.279212 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-8d7cb382-a0b5-4ebf-86ba-07e652515e4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9b798fa233243d19a7910ff98075abf', 'request_id': 'req-8d7cb382-a0b5-4ebf-86ba-07e652515e4a'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:31.292662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-8d7cb382-a0b5-4ebf-86ba-07e652515e4a admin admin] The token cache is being invalidate because project a9b798fa233243d19a7910ff98075abf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:31.292904 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-8d7cb382-a0b5-4ebf-86ba-07e652515e4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:31.293569 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-8d7cb382-a0b5-4ebf-86ba-07e652515e4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:31.295547 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 898/1805] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:31 2022] DELETE /identity/v3/projects/a9b798fa233243d19a7910ff98075abf => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:31.305750 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c2868-dbda-4029-9a4d-302f39b41cd9 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:31.306037 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c2868-dbda-4029-9a4d-302f39b41cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:31.306316 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c2868-dbda-4029-9a4d-302f39b41cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:31.325315 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-a58c2868-dbda-4029-9a4d-302f39b41cd9 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:31.326529 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-a58c2868-dbda-4029-9a4d-302f39b41cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:31.348620 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea015b-8931-4628-a0cc-958d64c55cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:31.350521 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea015b-8931-4628-a0cc-958d64c55cce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:31.351111 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea015b-8931-4628-a0cc-958d64c55cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:31.351425 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea015b-8931-4628-a0cc-958d64c55cce admin admin] PATH_INFO: `/v3/projects/49cd9de2cbef47a1b4ead8d74b0af9cc` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:31.353400 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ea015b-8931-4628-a0cc-958d64c55cce admin admin] RBAC: Authorizing `identity:delete_project(project_id=49cd9de2cbef47a1b4ead8d74b0af9cc)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:31.356619 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ea015b-8931-4628-a0cc-958d64c55cce admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:31.362175 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:31.375160 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:31.398425 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 899/1806] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:31.406441 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d89938-c651-49f8-9c18-6394c5311f04 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:31.406731 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d89938-c651-49f8-9c18-6394c5311f04 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:31.407103 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d89938-c651-49f8-9c18-6394c5311f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:31.425626 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-a4d89938-c651-49f8-9c18-6394c5311f04 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:31.426823 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-a4d89938-c651-49f8-9c18-6394c5311f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:31.428437 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-84ea015b-8931-4628-a0cc-958d64c55cce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49cd9de2cbef47a1b4ead8d74b0af9cc', 'request_id': 'req-84ea015b-8931-4628-a0cc-958d64c55cce'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:31.437071 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-84ea015b-8931-4628-a0cc-958d64c55cce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49cd9de2cbef47a1b4ead8d74b0af9cc', 'request_id': 'req-84ea015b-8931-4628-a0cc-958d64c55cce'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:31.444141 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-84ea015b-8931-4628-a0cc-958d64c55cce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49cd9de2cbef47a1b4ead8d74b0af9cc', 'request_id': 'req-84ea015b-8931-4628-a0cc-958d64c55cce'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:31.455171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-84ea015b-8931-4628-a0cc-958d64c55cce admin admin] The token cache is being invalidate because project 49cd9de2cbef47a1b4ead8d74b0af9cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:31.455466 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-84ea015b-8931-4628-a0cc-958d64c55cce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:31.456137 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-84ea015b-8931-4628-a0cc-958d64c55cce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:31.457887 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 908/1807] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:31 2022] DELETE /identity/v3/projects/49cd9de2cbef47a1b4ead8d74b0af9cc => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:31.463964 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb7e35-6b8a-482d-8dfb-78218f72e090 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:31.464259 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb7e35-6b8a-482d-8dfb-78218f72e090 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:31.464546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb7e35-6b8a-482d-8dfb-78218f72e090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:31.483932 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-9dbb7e35-6b8a-482d-8dfb-78218f72e090 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:31.485185 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-9dbb7e35-6b8a-482d-8dfb-78218f72e090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:31.495617 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 900/1808] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:43:31.500067 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d89938-c651-49f8-9c18-6394c5311f04 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:31.511102 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d89938-c651-49f8-9c18-6394c5311f04 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:31.515945 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d883805-4104-4d41-aa67-b3de36c14b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:31.516612 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d883805-4104-4d41-aa67-b3de36c14b89 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:31.516895 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d883805-4104-4d41-aa67-b3de36c14b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:31.517171 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d883805-4104-4d41-aa67-b3de36c14b89 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:31.517905 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d883805-4104-4d41-aa67-b3de36c14b89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:43:31.518253 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d883805-4104-4d41-aa67-b3de36c14b89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:31.520025 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d883805-4104-4d41-aa67-b3de36c14b89 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:31.533997 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 901/1809] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:43:31.538286 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d883805-4104-4d41-aa67-b3de36c14b89 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:31.549055 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d883805-4104-4d41-aa67-b3de36c14b89 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:31.554909 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f23f7-e1b0-4f85-8755-827747d8c3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:31.555801 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f23f7-e1b0-4f85-8755-827747d8c3c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:31.556305 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f23f7-e1b0-4f85-8755-827747d8c3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:31.556945 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 909/1810] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:43:31.558535 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f23f7-e1b0-4f85-8755-827747d8c3c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:31.559522 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f23f7-e1b0-4f85-8755-827747d8c3c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:31.561791 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f23f7-e1b0-4f85-8755-827747d8c3c2 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:31.565652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d1001-5dad-4fd1-8820-8e09ad111f40 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:31.566019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d1001-5dad-4fd1-8820-8e09ad111f40 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:31.566374 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d1001-5dad-4fd1-8820-8e09ad111f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:31.586673 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-e02d1001-5dad-4fd1-8820-8e09ad111f40 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:31.589772 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e02d1001-5dad-4fd1-8820-8e09ad111f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:31.598245 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 902/1811] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:31 2022] POST /identity/v3/projects => generated 387 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:43:31.602579 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f23f7-e1b0-4f85-8755-827747d8c3c2 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:31.669837 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 910/1812] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:43:31.676066 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d1001-5dad-4fd1-8820-8e09ad111f40 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:31.687966 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d1001-5dad-4fd1-8820-8e09ad111f40 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:31.688699 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f23f7-e1b0-4f85-8755-827747d8c3c2 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:31.695150 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc17778-804e-484e-8c7e-30231b89e9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:31.695468 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5ec993-ea8b-4295-8a22-37e97bed1190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:31.695824 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc17778-804e-484e-8c7e-30231b89e9f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:31.696153 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ec993-ea8b-4295-8a22-37e97bed1190 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:31.696273 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc17778-804e-484e-8c7e-30231b89e9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:31.696426 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc17778-804e-484e-8c7e-30231b89e9f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:31.696515 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ec993-ea8b-4295-8a22-37e97bed1190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:31.696710 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ec993-ea8b-4295-8a22-37e97bed1190 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:31.697153 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc17778-804e-484e-8c7e-30231b89e9f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:43:31.697408 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5ec993-ea8b-4295-8a22-37e97bed1190 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:31.697529 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc17778-804e-484e-8c7e-30231b89e9f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:31.699403 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc17778-804e-484e-8c7e-30231b89e9f2 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:31.703152 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5ec993-ea8b-4295-8a22-37e97bed1190 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:31.719410 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 911/1813] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:43:31.721934 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc17778-804e-484e-8c7e-30231b89e9f2 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:31.734272 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 903/1814] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:31 2022] POST /identity/v3/users => generated 294 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:31.734913 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc17778-804e-484e-8c7e-30231b89e9f2 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:31.735457 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5ec993-ea8b-4295-8a22-37e97bed1190 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:31.741419 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3e2d5-7c31-46ba-b232-ceabf4e6979d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:31.742093 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3e2d5-7c31-46ba-b232-ceabf4e6979d admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:31.742380 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3e2d5-7c31-46ba-b232-ceabf4e6979d admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:31.742654 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3e2d5-7c31-46ba-b232-ceabf4e6979d admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:31.743382 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad3e2d5-7c31-46ba-b232-ceabf4e6979d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:31.745933 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad3e2d5-7c31-46ba-b232-ceabf4e6979d admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:31.765778 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5ec993-ea8b-4295-8a22-37e97bed1190 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:31.772893 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c89342e-6d3c-4637-b70f-44f6e0b2bb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:31.773894 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89342e-6d3c-4637-b70f-44f6e0b2bb60 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:31.773894 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89342e-6d3c-4637-b70f-44f6e0b2bb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:31.774125 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89342e-6d3c-4637-b70f-44f6e0b2bb60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:31.775018 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c89342e-6d3c-4637-b70f-44f6e0b2bb60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:31.778294 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c89342e-6d3c-4637-b70f-44f6e0b2bb60 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:31.786158 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 912/1815] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:31 2022] POST /identity/v3/projects => generated 383 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:43:31.792156 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3e2d5-7c31-46ba-b232-ceabf4e6979d admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:31.794435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 904/1816] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:43:31.797485 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c89342e-6d3c-4637-b70f-44f6e0b2bb60 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:31.798942 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:31.817477 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c89342e-6d3c-4637-b70f-44f6e0b2bb60 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:31.855670 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:31.902529 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3e2d5-7c31-46ba-b232-ceabf4e6979d admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:31.908946 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1eb5db-1cd1-4702-b0a2-b5dbf3b34cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:31.909789 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1eb5db-1cd1-4702-b0a2-b5dbf3b34cea admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:31.910146 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1eb5db-1cd1-4702-b0a2-b5dbf3b34cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:31.910499 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1eb5db-1cd1-4702-b0a2-b5dbf3b34cea admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:31.911443 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1eb5db-1cd1-4702-b0a2-b5dbf3b34cea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:31.914028 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1eb5db-1cd1-4702-b0a2-b5dbf3b34cea admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:31.916631 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc1750-d178-4192-b113-a760bc3ec6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:31.917419 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc1750-d178-4192-b113-a760bc3ec6e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:31.917839 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc1750-d178-4192-b113-a760bc3ec6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:31.918183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc1750-d178-4192-b113-a760bc3ec6e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:31.919998 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc1750-d178-4192-b113-a760bc3ec6e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:31.923366 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc1750-d178-4192-b113-a760bc3ec6e1 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:31.938480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 913/1817] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:31 2022] POST /identity/v3/users => generated 292 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:31.942410 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 905/1818] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:31.944021 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc1750-d178-4192-b113-a760bc3ec6e1 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:31.944170 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1eb5db-1cd1-4702-b0a2-b5dbf3b34cea admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:31.955952 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1eb5db-1cd1-4702-b0a2-b5dbf3b34cea admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:31.957114 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc1750-d178-4192-b113-a760bc3ec6e1 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:31.961960 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b434fb-39dc-4b17-880f-55504d98f71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:31.962514 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b434fb-39dc-4b17-880f-55504d98f71e admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:31.962874 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b434fb-39dc-4b17-880f-55504d98f71e admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:31.963320 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b434fb-39dc-4b17-880f-55504d98f71e admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:31.964047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4609be-d935-4b5d-a510-60e8816839fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:31.964391 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b434fb-39dc-4b17-880f-55504d98f71e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:31.965169 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4609be-d935-4b5d-a510-60e8816839fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:31.965521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4609be-d935-4b5d-a510-60e8816839fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:31.965977 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4609be-d935-4b5d-a510-60e8816839fd admin admin] PATH_INFO: `/v3/projects/903457ce3f8d4882ba7f91f30b349b46/users/26a57c997860427db24d2e49f1b05e43/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:31.966293 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b434fb-39dc-4b17-880f-55504d98f71e admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:31.989152 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4609be-d935-4b5d-a510-60e8816839fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=903457ce3f8d4882ba7f91f30b349b46, user_id=26a57c997860427db24d2e49f1b05e43, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:31.992400 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4609be-d935-4b5d-a510-60e8816839fd admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:31.999364 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 914/1819] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:32.003791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b434fb-39dc-4b17-880f-55504d98f71e admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:32.020675 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b434fb-39dc-4b17-880f-55504d98f71e admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:32.021507 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 906/1820] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:31 2022] PUT /identity/v3/projects/903457ce3f8d4882ba7f91f30b349b46/users/26a57c997860427db24d2e49f1b05e43/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:32.023878 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d215ac-2071-4870-99da-8572d9083597 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:32.024174 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d215ac-2071-4870-99da-8572d9083597 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:32.024462 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d215ac-2071-4870-99da-8572d9083597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:32.059552 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-28d215ac-2071-4870-99da-8572d9083597 None None] MFA Rules not processed for user `26a57c997860427db24d2e49f1b05e43`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:32.061336 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-28d215ac-2071-4870-99da-8572d9083597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:32.092113 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4a596-b044-4fd4-8cd0-82f8c151f299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:32.092984 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4a596-b044-4fd4-8cd0-82f8c151f299 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:32.093096 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4a596-b044-4fd4-8cd0-82f8c151f299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:32.093354 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4a596-b044-4fd4-8cd0-82f8c151f299 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:32.094411 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff4a596-b044-4fd4-8cd0-82f8c151f299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:32.096070 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff4a596-b044-4fd4-8cd0-82f8c151f299 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:32.110957 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 915/1821] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:43:32.112183 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4a596-b044-4fd4-8cd0-82f8c151f299 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:32.125170 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4a596-b044-4fd4-8cd0-82f8c151f299 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:32.131275 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00228184-c821-421a-9e6c-ac010c7a8050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:32.132429 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00228184-c821-421a-9e6c-ac010c7a8050 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:32.132804 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00228184-c821-421a-9e6c-ac010c7a8050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:32.133145 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00228184-c821-421a-9e6c-ac010c7a8050 admin admin] PATH_INFO: `/v3/projects/bfaa73656ee442a29e00013c1ce3b5d6/users/6c18cff85c8747f6902524ac29a284bb/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:32.150263 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00228184-c821-421a-9e6c-ac010c7a8050 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfaa73656ee442a29e00013c1ce3b5d6, user_id=6c18cff85c8747f6902524ac29a284bb, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:32.153213 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00228184-c821-421a-9e6c-ac010c7a8050 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:32.155283 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 907/1822] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:32.159732 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d215ac-2071-4870-99da-8572d9083597 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:32.173685 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 916/1823] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:32 2022] PUT /identity/v3/projects/bfaa73656ee442a29e00013c1ce3b5d6/users/6c18cff85c8747f6902524ac29a284bb/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:43:32.186816 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d215ac-2071-4870-99da-8572d9083597 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:32.187658 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc145f7-5b6f-4746-8e9b-7aaa105349a9 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:32.187948 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc145f7-5b6f-4746-8e9b-7aaa105349a9 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:32.188354 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc145f7-5b6f-4746-8e9b-7aaa105349a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:32.217641 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-4bc145f7-5b6f-4746-8e9b-7aaa105349a9 None None] MFA Rules not processed for user `6c18cff85c8747f6902524ac29a284bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:32.218949 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-4bc145f7-5b6f-4746-8e9b-7aaa105349a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:32.256468 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacabecb-36b5-4db3-a3a3-9b3f687282eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:32.257314 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacabecb-36b5-4db3-a3a3-9b3f687282eb admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:32.257662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacabecb-36b5-4db3-a3a3-9b3f687282eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:32.258039 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacabecb-36b5-4db3-a3a3-9b3f687282eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:32.258813 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacabecb-36b5-4db3-a3a3-9b3f687282eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:32.261054 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacabecb-36b5-4db3-a3a3-9b3f687282eb admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:32.284107 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:32.294176 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 908/1824] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:32 2022] POST /identity/v3/projects => generated 387 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:43:32.295208 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacabecb-36b5-4db3-a3a3-9b3f687282eb admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:32.306773 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:32.311461 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 917/1825] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:32.323419 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc145f7-5b6f-4746-8e9b-7aaa105349a9 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:32.324860 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-4bc145f7-5b6f-4746-8e9b-7aaa105349a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:32.364341 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacabecb-36b5-4db3-a3a3-9b3f687282eb admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:32.369870 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a18c065-fab9-4a6f-93e6-89142b2d8cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:32.370688 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18c065-fab9-4a6f-93e6-89142b2d8cc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:32.371040 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18c065-fab9-4a6f-93e6-89142b2d8cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:32.371335 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18c065-fab9-4a6f-93e6-89142b2d8cc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:32.372043 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a18c065-fab9-4a6f-93e6-89142b2d8cc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:32.373968 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a18c065-fab9-4a6f-93e6-89142b2d8cc7 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:32.383091 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc145f7-5b6f-4746-8e9b-7aaa105349a9 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:32.395119 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 909/1826] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:32 2022] POST /identity/v3/users => generated 300 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:43:32.399936 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a18c065-fab9-4a6f-93e6-89142b2d8cc7 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:32.410655 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a18c065-fab9-4a6f-93e6-89142b2d8cc7 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:32.417265 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ba303-5e17-4b2f-9ae6-f93213a13bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:32.417954 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ba303-5e17-4b2f-9ae6-f93213a13bae admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:32.418230 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ba303-5e17-4b2f-9ae6-f93213a13bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:32.418510 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ba303-5e17-4b2f-9ae6-f93213a13bae admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:32.419570 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ba303-5e17-4b2f-9ae6-f93213a13bae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:32.421171 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ba303-5e17-4b2f-9ae6-f93213a13bae admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:32.433391 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 910/1827] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:43:32.438303 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ba303-5e17-4b2f-9ae6-f93213a13bae admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:32.448870 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ba303-5e17-4b2f-9ae6-f93213a13bae admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:32.450791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac4bd3-eb2a-437d-9493-90e73a015cf3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:32.451643 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac4bd3-eb2a-437d-9493-90e73a015cf3 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:32.451929 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac4bd3-eb2a-437d-9493-90e73a015cf3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:32.452204 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac4bd3-eb2a-437d-9493-90e73a015cf3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:32.454624 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af019061-4dba-4222-a32a-7b552abe6b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:32.455616 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af019061-4dba-4222-a32a-7b552abe6b85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:32.455918 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af019061-4dba-4222-a32a-7b552abe6b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:32.456277 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af019061-4dba-4222-a32a-7b552abe6b85 admin admin] PATH_INFO: `/v3/projects/cd53a7e5faad4db980d6e67f0042915b/users/bb044e6f0411467296eba132c956a2a2/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:32.473815 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af019061-4dba-4222-a32a-7b552abe6b85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd53a7e5faad4db980d6e67f0042915b, user_id=bb044e6f0411467296eba132c956a2a2, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:32.477275 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af019061-4dba-4222-a32a-7b552abe6b85 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:32.499726 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 911/1828] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:32 2022] PUT /identity/v3/projects/cd53a7e5faad4db980d6e67f0042915b/users/bb044e6f0411467296eba132c956a2a2/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:43:32.501045 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af019061-4dba-4222-a32a-7b552abe6b85 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:32.550170 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ac4bd3-eb2a-437d-9493-90e73a015cf3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:32.553323 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ac4bd3-eb2a-437d-9493-90e73a015cf3 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:32.565561 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 918/1829] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:43:32.596573 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af019061-4dba-4222-a32a-7b552abe6b85 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:32.602096 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf438e0-57b2-4791-b765-77130aee76bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:32.602903 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf438e0-57b2-4791-b765-77130aee76bc admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:32.603245 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf438e0-57b2-4791-b765-77130aee76bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:32.603528 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf438e0-57b2-4791-b765-77130aee76bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:32.604272 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf438e0-57b2-4791-b765-77130aee76bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:32.605884 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf438e0-57b2-4791-b765-77130aee76bc admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:32.617530 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 912/1830] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:43:32.622041 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac4bd3-eb2a-437d-9493-90e73a015cf3 service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:32.635734 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac4bd3-eb2a-437d-9493-90e73a015cf3 service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:32.640824 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4400064c-4853-4a8a-8adf-3079ff25a6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:32.641856 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400064c-4853-4a8a-8adf-3079ff25a6dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:32.642144 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400064c-4853-4a8a-8adf-3079ff25a6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:32.642420 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400064c-4853-4a8a-8adf-3079ff25a6dc admin admin] PATH_INFO: `/v3/projects/cd53a7e5faad4db980d6e67f0042915b/users/bb044e6f0411467296eba132c956a2a2/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:32.644678 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4400064c-4853-4a8a-8adf-3079ff25a6dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd53a7e5faad4db980d6e67f0042915b, user_id=bb044e6f0411467296eba132c956a2a2, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:32.648006 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4400064c-4853-4a8a-8adf-3079ff25a6dc admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:32.659933 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 919/1831] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:32 2022] PUT /identity/v3/projects/cd53a7e5faad4db980d6e67f0042915b/users/bb044e6f0411467296eba132c956a2a2/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:32.672099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48575da-d071-47c6-b9b4-c1f2021f2aa4 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:32.672442 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48575da-d071-47c6-b9b4-c1f2021f2aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:32.672774 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48575da-d071-47c6-b9b4-c1f2021f2aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:32.735640 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-c48575da-d071-47c6-b9b4-c1f2021f2aa4 None None] MFA Rules not processed for user `bb044e6f0411467296eba132c956a2a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:32.738784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c48575da-d071-47c6-b9b4-c1f2021f2aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:32.860666 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:32.894680 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 913/1832] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:32 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:32.906342 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4400064c-4853-4a8a-8adf-3079ff25a6dc admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:32.909793 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-4400064c-4853-4a8a-8adf-3079ff25a6dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:33.041760 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:33.058460 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4400064c-4853-4a8a-8adf-3079ff25a6dc admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:33.063423 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb172c79-4f80-4fdb-a773-48386c5d8827 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:33.064169 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb172c79-4f80-4fdb-a773-48386c5d8827 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:33.065041 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb172c79-4f80-4fdb-a773-48386c5d8827 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:33.065041 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb172c79-4f80-4fdb-a773-48386c5d8827 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:33.084903 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb172c79-4f80-4fdb-a773-48386c5d8827 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:33.086644 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb172c79-4f80-4fdb-a773-48386c5d8827 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:33.098379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 920/1833] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:32 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:33.264135 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48575da-d071-47c6-b9b4-c1f2021f2aa4 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:33.380473 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48575da-d071-47c6-b9b4-c1f2021f2aa4 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:33.387768 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf189c-0a8a-4049-a70a-58b84ce13989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:33.388653 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf189c-0a8a-4049-a70a-58b84ce13989 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:33.389063 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf189c-0a8a-4049-a70a-58b84ce13989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:33.391276 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf189c-0a8a-4049-a70a-58b84ce13989 admin admin] PATH_INFO: `/v3/users/26a57c997860427db24d2e49f1b05e43` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:33.391276 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf189c-0a8a-4049-a70a-58b84ce13989 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26a57c997860427db24d2e49f1b05e43)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:33.392810 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf189c-0a8a-4049-a70a-58b84ce13989 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:33.459563 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6edf189c-0a8a-4049-a70a-58b84ce13989 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26a57c997860427db24d2e49f1b05e43'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:33.460966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6edf189c-0a8a-4049-a70a-58b84ce13989 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26a57c997860427db24d2e49f1b05e43'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:33.491381 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6edf189c-0a8a-4049-a70a-58b84ce13989 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26a57c997860427db24d2e49f1b05e43'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:33.504503 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6edf189c-0a8a-4049-a70a-58b84ce13989 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26a57c997860427db24d2e49f1b05e43'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:33.512908 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6edf189c-0a8a-4049-a70a-58b84ce13989 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26a57c997860427db24d2e49f1b05e43'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:33.519629 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 914/1834] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:33 2022] DELETE /identity/v3/users/26a57c997860427db24d2e49f1b05e43 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:33.524689 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6adc12-02a1-4ee2-9335-98f559859ccf None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:33.525230 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6adc12-02a1-4ee2-9335-98f559859ccf None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:33.525230 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6adc12-02a1-4ee2-9335-98f559859ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:33.545484 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-2a6adc12-02a1-4ee2-9335-98f559859ccf None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:33.546694 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-2a6adc12-02a1-4ee2-9335-98f559859ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:33.616869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 921/1835] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:33.626738 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf189c-0a8a-4049-a70a-58b84ce13989 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:33.628249 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-6edf189c-0a8a-4049-a70a-58b84ce13989 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:33.694534 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf189c-0a8a-4049-a70a-58b84ce13989 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:33.698197 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6ae39-9fd3-447f-ae4a-d3d4adce5274 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:33.698862 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6ae39-9fd3-447f-ae4a-d3d4adce5274 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:33.699186 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6ae39-9fd3-447f-ae4a-d3d4adce5274 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:33.699471 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6ae39-9fd3-447f-ae4a-d3d4adce5274 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:33.710640 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a6ae39-9fd3-447f-ae4a-d3d4adce5274 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:33.712152 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a6ae39-9fd3-447f-ae4a-d3d4adce5274 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:33.722867 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 915/1836] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:34.019144 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6adc12-02a1-4ee2-9335-98f559859ccf None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:34.021431 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-2a6adc12-02a1-4ee2-9335-98f559859ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:34.043754 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6adc12-02a1-4ee2-9335-98f559859ccf None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:34.049734 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32776b-0f1a-4f6c-abcc-5635bf65680b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:34.050608 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32776b-0f1a-4f6c-abcc-5635bf65680b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:34.050908 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32776b-0f1a-4f6c-abcc-5635bf65680b admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:34.051353 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32776b-0f1a-4f6c-abcc-5635bf65680b admin admin] PATH_INFO: `/v3/projects/903457ce3f8d4882ba7f91f30b349b46` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:34.053240 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b32776b-0f1a-4f6c-abcc-5635bf65680b admin admin] RBAC: Authorizing `identity:delete_project(project_id=903457ce3f8d4882ba7f91f30b349b46)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:34.054880 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b32776b-0f1a-4f6c-abcc-5635bf65680b admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:34.121011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-7b32776b-0f1a-4f6c-abcc-5635bf65680b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '903457ce3f8d4882ba7f91f30b349b46', 'request_id': 'req-7b32776b-0f1a-4f6c-abcc-5635bf65680b'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:34.129738 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-7b32776b-0f1a-4f6c-abcc-5635bf65680b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '903457ce3f8d4882ba7f91f30b349b46', 'request_id': 'req-7b32776b-0f1a-4f6c-abcc-5635bf65680b'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:34.137372 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-7b32776b-0f1a-4f6c-abcc-5635bf65680b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '903457ce3f8d4882ba7f91f30b349b46', 'request_id': 'req-7b32776b-0f1a-4f6c-abcc-5635bf65680b'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:34.154489 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-7b32776b-0f1a-4f6c-abcc-5635bf65680b admin admin] The token cache is being invalidate because project 903457ce3f8d4882ba7f91f30b349b46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:34.154809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-7b32776b-0f1a-4f6c-abcc-5635bf65680b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:34.156492 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-7b32776b-0f1a-4f6c-abcc-5635bf65680b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:34.158560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 922/1837] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:34 2022] DELETE /identity/v3/projects/903457ce3f8d4882ba7f91f30b349b46 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:34.163185 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6ae39-9fd3-447f-ae4a-d3d4adce5274 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:34.173446 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6ae39-9fd3-447f-ae4a-d3d4adce5274 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:34.178703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:34.292559 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f99db3-d8e8-4bb6-ba40-29aba61bb32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:34.293442 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f99db3-d8e8-4bb6-ba40-29aba61bb32f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:34.293723 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f99db3-d8e8-4bb6-ba40-29aba61bb32f admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:34.293996 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f99db3-d8e8-4bb6-ba40-29aba61bb32f admin admin] PATH_INFO: `/v3/users/bb044e6f0411467296eba132c956a2a2` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:34.295234 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f99db3-d8e8-4bb6-ba40-29aba61bb32f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb044e6f0411467296eba132c956a2a2)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:34.297141 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f99db3-d8e8-4bb6-ba40-29aba61bb32f admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:34.348675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-05f99db3-d8e8-4bb6-ba40-29aba61bb32f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb044e6f0411467296eba132c956a2a2'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:34.350671 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-05f99db3-d8e8-4bb6-ba40-29aba61bb32f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb044e6f0411467296eba132c956a2a2'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:34.362909 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-05f99db3-d8e8-4bb6-ba40-29aba61bb32f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb044e6f0411467296eba132c956a2a2'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:34.373257 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-05f99db3-d8e8-4bb6-ba40-29aba61bb32f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb044e6f0411467296eba132c956a2a2'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:34.381716 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-05f99db3-d8e8-4bb6-ba40-29aba61bb32f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb044e6f0411467296eba132c956a2a2'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:34.388167 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 916/1838] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:34 2022] DELETE /identity/v3/users/bb044e6f0411467296eba132c956a2a2 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:34.737076 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32776b-0f1a-4f6c-abcc-5635bf65680b admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:34.738860 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-7b32776b-0f1a-4f6c-abcc-5635bf65680b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:34.752609 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32776b-0f1a-4f6c-abcc-5635bf65680b admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:34.821753 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3b328b-a2ee-4872-b3bd-b88e2181fc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:34.822651 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b328b-a2ee-4872-b3bd-b88e2181fc07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:34.822930 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b328b-a2ee-4872-b3bd-b88e2181fc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:34.823266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b328b-a2ee-4872-b3bd-b88e2181fc07 admin admin] PATH_INFO: `/v3/projects/cd53a7e5faad4db980d6e67f0042915b` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:34.824521 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3b328b-a2ee-4872-b3bd-b88e2181fc07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd53a7e5faad4db980d6e67f0042915b)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:34.826495 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3b328b-a2ee-4872-b3bd-b88e2181fc07 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:34.840621 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:34.880470 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-bb3b328b-a2ee-4872-b3bd-b88e2181fc07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd53a7e5faad4db980d6e67f0042915b', 'request_id': 'req-bb3b328b-a2ee-4872-b3bd-b88e2181fc07'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:34.888714 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-bb3b328b-a2ee-4872-b3bd-b88e2181fc07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd53a7e5faad4db980d6e67f0042915b', 'request_id': 'req-bb3b328b-a2ee-4872-b3bd-b88e2181fc07'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:34.895241 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-bb3b328b-a2ee-4872-b3bd-b88e2181fc07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd53a7e5faad4db980d6e67f0042915b', 'request_id': 'req-bb3b328b-a2ee-4872-b3bd-b88e2181fc07'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:34.906637 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-bb3b328b-a2ee-4872-b3bd-b88e2181fc07 admin admin] The token cache is being invalidate because project cd53a7e5faad4db980d6e67f0042915b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:34.906924 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-bb3b328b-a2ee-4872-b3bd-b88e2181fc07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:34.907610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-bb3b328b-a2ee-4872-b3bd-b88e2181fc07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:34.909212 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 923/1839] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:34 2022] DELETE /identity/v3/projects/cd53a7e5faad4db980d6e67f0042915b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:34.915410 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f209c91-9a4f-444d-a172-32aec4dba391 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:34.915703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f209c91-9a4f-444d-a172-32aec4dba391 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:34.915986 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f209c91-9a4f-444d-a172-32aec4dba391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:34.943411 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-1f209c91-9a4f-444d-a172-32aec4dba391 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:34.944686 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1f209c91-9a4f-444d-a172-32aec4dba391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:35.003331 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 917/1840] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:35.011552 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58679c9-fe71-44dc-b50f-04f5d6cb6a01 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:35.011850 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58679c9-fe71-44dc-b50f-04f5d6cb6a01 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:35.012136 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58679c9-fe71-44dc-b50f-04f5d6cb6a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:35.031491 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-e58679c9-fe71-44dc-b50f-04f5d6cb6a01 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:35.032712 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e58679c9-fe71-44dc-b50f-04f5d6cb6a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:35.041983 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 924/1841] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:35.046455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f209c91-9a4f-444d-a172-32aec4dba391 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:35.056750 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f209c91-9a4f-444d-a172-32aec4dba391 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:35.061326 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7017d7a2-f014-4b53-abce-fc3235202498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:35.061982 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017d7a2-f014-4b53-abce-fc3235202498 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:35.062265 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017d7a2-f014-4b53-abce-fc3235202498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:35.062545 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017d7a2-f014-4b53-abce-fc3235202498 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:35.063300 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7017d7a2-f014-4b53-abce-fc3235202498 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:43:35.063611 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7017d7a2-f014-4b53-abce-fc3235202498 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:35.065211 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7017d7a2-f014-4b53-abce-fc3235202498 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:35.079551 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 918/1842] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:43:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:43:35.084560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58679c9-fe71-44dc-b50f-04f5d6cb6a01 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:35.093823 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58679c9-fe71-44dc-b50f-04f5d6cb6a01 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:35.098514 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded1389-d10c-49b9-8c23-d2236264645e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:35.099206 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded1389-d10c-49b9-8c23-d2236264645e admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:35.099493 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded1389-d10c-49b9-8c23-d2236264645e admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:35.099771 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded1389-d10c-49b9-8c23-d2236264645e admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:35.100465 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded1389-d10c-49b9-8c23-d2236264645e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:35.102362 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded1389-d10c-49b9-8c23-d2236264645e admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:35.128302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 925/1843] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:35 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:35.132778 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7017d7a2-f014-4b53-abce-fc3235202498 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:35.142712 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7017d7a2-f014-4b53-abce-fc3235202498 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:35.152191 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:35.204309 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce348cd-f401-4976-abfe-99835cd6fd19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:35.204992 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce348cd-f401-4976-abfe-99835cd6fd19 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:35.205280 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce348cd-f401-4976-abfe-99835cd6fd19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:35.205567 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce348cd-f401-4976-abfe-99835cd6fd19 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:35.206261 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce348cd-f401-4976-abfe-99835cd6fd19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:35.208844 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce348cd-f401-4976-abfe-99835cd6fd19 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:35.230649 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 919/1844] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:35 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:35.234717 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded1389-d10c-49b9-8c23-d2236264645e admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:35.245731 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded1389-d10c-49b9-8c23-d2236264645e admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:35.251018 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763bb880-f0d3-4fbb-9e1e-8de1a27319a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:35.251705 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763bb880-f0d3-4fbb-9e1e-8de1a27319a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:35.251992 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763bb880-f0d3-4fbb-9e1e-8de1a27319a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:35.252276 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763bb880-f0d3-4fbb-9e1e-8de1a27319a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:35.253016 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763bb880-f0d3-4fbb-9e1e-8de1a27319a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:35.254909 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763bb880-f0d3-4fbb-9e1e-8de1a27319a9 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:35.269620 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 926/1845] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:35.270856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce348cd-f401-4976-abfe-99835cd6fd19 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:35.281694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce348cd-f401-4976-abfe-99835cd6fd19 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:35.286968 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78cb268-10fc-456d-ba4f-f21ee44985b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:35.287656 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78cb268-10fc-456d-ba4f-f21ee44985b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:35.287940 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78cb268-10fc-456d-ba4f-f21ee44985b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:35.288213 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78cb268-10fc-456d-ba4f-f21ee44985b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:35.288933 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78cb268-10fc-456d-ba4f-f21ee44985b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:35.290516 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78cb268-10fc-456d-ba4f-f21ee44985b5 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:35.301106 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 920/1846] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:35.305912 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763bb880-f0d3-4fbb-9e1e-8de1a27319a9 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:35.317539 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763bb880-f0d3-4fbb-9e1e-8de1a27319a9 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:35.322067 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9012d0-b922-44aa-b89a-6399be0ac43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:35.322969 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9012d0-b922-44aa-b89a-6399be0ac43a admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:35.323290 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9012d0-b922-44aa-b89a-6399be0ac43a admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:35.323572 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9012d0-b922-44aa-b89a-6399be0ac43a admin admin] PATH_INFO: `/v3/projects/f34a66af437543cf8f886bce61210759/users/82c24de3fe5d48029801df2127b24590/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:35.339844 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9012d0-b922-44aa-b89a-6399be0ac43a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f34a66af437543cf8f886bce61210759, user_id=82c24de3fe5d48029801df2127b24590, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:35.342752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9012d0-b922-44aa-b89a-6399be0ac43a admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:35.357370 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 927/1847] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:35 2022] PUT /identity/v3/projects/f34a66af437543cf8f886bce61210759/users/82c24de3fe5d48029801df2127b24590/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:35.369255 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c7e6d-ef3a-4a13-ab7f-6fc7982725e1 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:35.369544 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c7e6d-ef3a-4a13-ab7f-6fc7982725e1 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:35.369822 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c7e6d-ef3a-4a13-ab7f-6fc7982725e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:35.398283 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-373c7e6d-ef3a-4a13-ab7f-6fc7982725e1 None None] MFA Rules not processed for user `82c24de3fe5d48029801df2127b24590`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:35.399483 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-373c7e6d-ef3a-4a13-ab7f-6fc7982725e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:35.468156 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 921/1848] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:35.472705 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9012d0-b922-44aa-b89a-6399be0ac43a admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:35.564272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9012d0-b922-44aa-b89a-6399be0ac43a admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:35.570538 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad73dc-b886-45ac-9d7b-72aae8bd2045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:35.571389 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad73dc-b886-45ac-9d7b-72aae8bd2045 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:35.571838 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad73dc-b886-45ac-9d7b-72aae8bd2045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:35.572133 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad73dc-b886-45ac-9d7b-72aae8bd2045 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:35.572854 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad73dc-b886-45ac-9d7b-72aae8bd2045 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:35.574976 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad73dc-b886-45ac-9d7b-72aae8bd2045 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:35.620461 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c7e6d-ef3a-4a13-ab7f-6fc7982725e1 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:35.622688 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 928/1849] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:35 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:35.637763 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c7e6d-ef3a-4a13-ab7f-6fc7982725e1 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:35.720657 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed873682-b942-4e7f-91ef-bcdf6dfc4a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:35.721304 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed873682-b942-4e7f-91ef-bcdf6dfc4a92 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:35.721602 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed873682-b942-4e7f-91ef-bcdf6dfc4a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:35.721893 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed873682-b942-4e7f-91ef-bcdf6dfc4a92 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:35.722633 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed873682-b942-4e7f-91ef-bcdf6dfc4a92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:35.725710 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed873682-b942-4e7f-91ef-bcdf6dfc4a92 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:35.762108 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 922/1850] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:35 2022] POST /identity/v3/users => generated 299 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:35.765923 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad73dc-b886-45ac-9d7b-72aae8bd2045 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:35.779458 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad73dc-b886-45ac-9d7b-72aae8bd2045 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:35.785091 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f25aaa4-fa5d-43de-8726-6e78c819b7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:35.785784 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25aaa4-fa5d-43de-8726-6e78c819b7e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:35.787349 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25aaa4-fa5d-43de-8726-6e78c819b7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:35.787349 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25aaa4-fa5d-43de-8726-6e78c819b7e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:35.787745 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f25aaa4-fa5d-43de-8726-6e78c819b7e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:35.790302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f25aaa4-fa5d-43de-8726-6e78c819b7e1 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:35.806338 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 929/1851] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:35.807340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed873682-b942-4e7f-91ef-bcdf6dfc4a92 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:35.825877 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed873682-b942-4e7f-91ef-bcdf6dfc4a92 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:35.831248 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275571a1-2efd-4fde-b4aa-7dc2c89684f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:35.832121 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275571a1-2efd-4fde-b4aa-7dc2c89684f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:35.832456 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275571a1-2efd-4fde-b4aa-7dc2c89684f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:35.832766 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275571a1-2efd-4fde-b4aa-7dc2c89684f3 admin admin] PATH_INFO: `/v3/projects/af7556007c6f4035bbe9f8be21099555/users/910d3360da1c414496c0eb30ce0e1293/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:35.844249 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:35.856111 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275571a1-2efd-4fde-b4aa-7dc2c89684f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af7556007c6f4035bbe9f8be21099555, user_id=910d3360da1c414496c0eb30ce0e1293, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:35.858945 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275571a1-2efd-4fde-b4aa-7dc2c89684f3 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:35.879056 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 923/1852] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:35 2022] PUT /identity/v3/projects/af7556007c6f4035bbe9f8be21099555/users/910d3360da1c414496c0eb30ce0e1293/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:35.883527 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f25aaa4-fa5d-43de-8726-6e78c819b7e1 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:35.904453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f25aaa4-fa5d-43de-8726-6e78c819b7e1 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:35.910155 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:36.002318 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef64e4-5218-4126-b992-93b6965547a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:36.003093 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef64e4-5218-4126-b992-93b6965547a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:36.003380 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef64e4-5218-4126-b992-93b6965547a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:36.003657 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef64e4-5218-4126-b992-93b6965547a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:36.004402 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ef64e4-5218-4126-b992-93b6965547a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:36.006946 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ef64e4-5218-4126-b992-93b6965547a0 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:36.019056 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 930/1853] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:43:36.023464 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275571a1-2efd-4fde-b4aa-7dc2c89684f3 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:36.035068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275571a1-2efd-4fde-b4aa-7dc2c89684f3 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:36.043606 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e57ce93-519b-4e5b-bf7c-2f10085aef16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:36.044754 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57ce93-519b-4e5b-bf7c-2f10085aef16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:36.045106 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57ce93-519b-4e5b-bf7c-2f10085aef16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:36.045527 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57ce93-519b-4e5b-bf7c-2f10085aef16 admin admin] PATH_INFO: `/v3/projects/af7556007c6f4035bbe9f8be21099555/users/910d3360da1c414496c0eb30ce0e1293/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:36.055684 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e57ce93-519b-4e5b-bf7c-2f10085aef16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af7556007c6f4035bbe9f8be21099555, user_id=910d3360da1c414496c0eb30ce0e1293, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:36.058678 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e57ce93-519b-4e5b-bf7c-2f10085aef16 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:36.074728 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 924/1854] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:36 2022] PUT /identity/v3/projects/af7556007c6f4035bbe9f8be21099555/users/910d3360da1c414496c0eb30ce0e1293/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:43:36.087226 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c95b9-bcc6-4ad6-8a32-a4861306ab3f None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:36.087546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c95b9-bcc6-4ad6-8a32-a4861306ab3f None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:36.087828 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c95b9-bcc6-4ad6-8a32-a4861306ab3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:36.116909 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-233c95b9-bcc6-4ad6-8a32-a4861306ab3f None None] MFA Rules not processed for user `910d3360da1c414496c0eb30ce0e1293`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:36.118148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-233c95b9-bcc6-4ad6-8a32-a4861306ab3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:36.203950 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 931/1855] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:36 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:36.209544 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5229ba62-d955-4725-9620-f0c23b1e9bbf None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:36.210413 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5229ba62-d955-4725-9620-f0c23b1e9bbf None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:36.210413 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5229ba62-d955-4725-9620-f0c23b1e9bbf None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:36.211549 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 925/1856] 10.222.0.45 () {54 vars in 921 bytes} [Wed Jun 15 14:43:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:43:36.220613 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233c95b9-bcc6-4ad6-8a32-a4861306ab3f None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:36.222070 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-233c95b9-bcc6-4ad6-8a32-a4861306ab3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:36.227435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffff25-af04-4e7b-bdd4-c7224784df47 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:36.227965 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffff25-af04-4e7b-bdd4-c7224784df47 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:36.228248 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffff25-af04-4e7b-bdd4-c7224784df47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:36.251100 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-67ffff25-af04-4e7b-bdd4-c7224784df47 None None] MFA Rules not processed for user `f3185bd2458c4e22830975700888e9be`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:36.252420 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-67ffff25-af04-4e7b-bdd4-c7224784df47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:36.303043 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233c95b9-bcc6-4ad6-8a32-a4861306ab3f None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:36.312433 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3acd05-d085-4520-8fb0-1fe61eeeb230 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:36.313528 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3acd05-d085-4520-8fb0-1fe61eeeb230 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:36.314008 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3acd05-d085-4520-8fb0-1fe61eeeb230 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:36.314487 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3acd05-d085-4520-8fb0-1fe61eeeb230 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:36.332520 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 926/1857] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Jun 15 14:43:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:36.346414 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffff25-af04-4e7b-bdd4-c7224784df47 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:36.348780 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-67ffff25-af04-4e7b-bdd4-c7224784df47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:36.363046 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffff25-af04-4e7b-bdd4-c7224784df47 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:36.370569 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab1c38e-0675-47b8-a9c7-ce749868bb58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:36.371264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab1c38e-0675-47b8-a9c7-ce749868bb58 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:36.371978 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab1c38e-0675-47b8-a9c7-ce749868bb58 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:36.372274 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab1c38e-0675-47b8-a9c7-ce749868bb58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:36.375687 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:36.383257 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab1c38e-0675-47b8-a9c7-ce749868bb58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:36.384851 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab1c38e-0675-47b8-a9c7-ce749868bb58 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:36.390332 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3acd05-d085-4520-8fb0-1fe61eeeb230 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:36.392971 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3acd05-d085-4520-8fb0-1fe61eeeb230 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:36.396391 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 927/1858] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:43:36.404701 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 932/1859] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:44.603646 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab1c38e-0675-47b8-a9c7-ce749868bb58 service nova] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:44.606612 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1ab1c38e-0675-47b8-a9c7-ce749868bb58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:44.737850 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab1c38e-0675-47b8-a9c7-ce749868bb58 service nova] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:44.750882 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c38c9-0e66-49fc-86f0-1574d1446ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:44.752156 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c38c9-0e66-49fc-86f0-1574d1446ca2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:44.752590 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c38c9-0e66-49fc-86f0-1574d1446ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:44.753167 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c38c9-0e66-49fc-86f0-1574d1446ca2 admin admin] PATH_INFO: `/v3/users/f53604bd286c4676a51e77582910de8f` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:44.755612 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7c38c9-0e66-49fc-86f0-1574d1446ca2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f53604bd286c4676a51e77582910de8f)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:44.757902 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7c38c9-0e66-49fc-86f0-1574d1446ca2 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:44.819784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-5d7c38c9-0e66-49fc-86f0-1574d1446ca2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f53604bd286c4676a51e77582910de8f'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:44.820443 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-5d7c38c9-0e66-49fc-86f0-1574d1446ca2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f53604bd286c4676a51e77582910de8f'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:44.826350 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:44.835949 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-5d7c38c9-0e66-49fc-86f0-1574d1446ca2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f53604bd286c4676a51e77582910de8f'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:44.844007 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-5d7c38c9-0e66-49fc-86f0-1574d1446ca2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f53604bd286c4676a51e77582910de8f'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:44.854562 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-5d7c38c9-0e66-49fc-86f0-1574d1446ca2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f53604bd286c4676a51e77582910de8f'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:44.861403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 928/1860] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:44 2022] DELETE /identity/v3/users/f53604bd286c4676a51e77582910de8f => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:44.866599 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab4aa6-df1a-49b1-90e4-bea15dbed6c1 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:44.867070 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab4aa6-df1a-49b1-90e4-bea15dbed6c1 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:44.867199 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab4aa6-df1a-49b1-90e4-bea15dbed6c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:44.889885 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-23ab4aa6-df1a-49b1-90e4-bea15dbed6c1 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:44.891160 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-23ab4aa6-df1a-49b1-90e4-bea15dbed6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:44.985859 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 933/1861] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:44.997743 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c38c9-0e66-49fc-86f0-1574d1446ca2 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:44.999257 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-5d7c38c9-0e66-49fc-86f0-1574d1446ca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:45.061429 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c38c9-0e66-49fc-86f0-1574d1446ca2 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:45.065613 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d312a-d27f-4b02-bd00-6872bb916aa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:45.066336 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d312a-d27f-4b02-bd00-6872bb916aa7 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:45.066647 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d312a-d27f-4b02-bd00-6872bb916aa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:45.066923 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d312a-d27f-4b02-bd00-6872bb916aa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:45.077480 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d312a-d27f-4b02-bd00-6872bb916aa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:45.078943 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d312a-d27f-4b02-bd00-6872bb916aa7 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:45.091973 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 929/1862] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:43:46.286938 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab4aa6-df1a-49b1-90e4-bea15dbed6c1 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:46.289454 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-23ab4aa6-df1a-49b1-90e4-bea15dbed6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:46.309576 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab4aa6-df1a-49b1-90e4-bea15dbed6c1 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:46.316821 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a5959-a4d0-4148-8a6b-6d6f2943bd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:46.318230 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a5959-a4d0-4148-8a6b-6d6f2943bd39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:46.319068 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a5959-a4d0-4148-8a6b-6d6f2943bd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:46.319555 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a5959-a4d0-4148-8a6b-6d6f2943bd39 admin admin] PATH_INFO: `/v3/projects/40a7c2e314ac4335b18f08359e7f66f2` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:46.321099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265a5959-a4d0-4148-8a6b-6d6f2943bd39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40a7c2e314ac4335b18f08359e7f66f2)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:46.323584 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265a5959-a4d0-4148-8a6b-6d6f2943bd39 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:46.414008 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-265a5959-a4d0-4148-8a6b-6d6f2943bd39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40a7c2e314ac4335b18f08359e7f66f2', 'request_id': 'req-265a5959-a4d0-4148-8a6b-6d6f2943bd39'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:46.425900 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-265a5959-a4d0-4148-8a6b-6d6f2943bd39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40a7c2e314ac4335b18f08359e7f66f2', 'request_id': 'req-265a5959-a4d0-4148-8a6b-6d6f2943bd39'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:46.436313 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-265a5959-a4d0-4148-8a6b-6d6f2943bd39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40a7c2e314ac4335b18f08359e7f66f2', 'request_id': 'req-265a5959-a4d0-4148-8a6b-6d6f2943bd39'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:46.452209 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-265a5959-a4d0-4148-8a6b-6d6f2943bd39 admin admin] The token cache is being invalidate because project 40a7c2e314ac4335b18f08359e7f66f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:46.452892 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-265a5959-a4d0-4148-8a6b-6d6f2943bd39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:46.453915 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-265a5959-a4d0-4148-8a6b-6d6f2943bd39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:46.456831 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 934/1863] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:46 2022] DELETE /identity/v3/projects/40a7c2e314ac4335b18f08359e7f66f2 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:46.462778 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d312a-d27f-4b02-bd00-6872bb916aa7 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:46.478536 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d312a-d27f-4b02-bd00-6872bb916aa7 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:46.480878 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a5959-a4d0-4148-8a6b-6d6f2943bd39 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:46.488443 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-265a5959-a4d0-4148-8a6b-6d6f2943bd39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:46.631243 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a5959-a4d0-4148-8a6b-6d6f2943bd39 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:46.636501 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d7612a-872e-4074-90d3-87498ccced56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:46.637586 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7612a-872e-4074-90d3-87498ccced56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:46.637870 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7612a-872e-4074-90d3-87498ccced56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:46.638175 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7612a-872e-4074-90d3-87498ccced56 admin admin] PATH_INFO: `/v3/users/4637bd688ad74a31a739759076a523d3` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:46.640163 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d7612a-872e-4074-90d3-87498ccced56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4637bd688ad74a31a739759076a523d3)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:46.640829 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c51d5-47fa-4beb-869b-32beea2bfd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:46.641729 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c51d5-47fa-4beb-869b-32beea2bfd9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:46.642015 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c51d5-47fa-4beb-869b-32beea2bfd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:46.642117 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d7612a-872e-4074-90d3-87498ccced56 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:46.642296 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c51d5-47fa-4beb-869b-32beea2bfd9c admin admin] PATH_INFO: `/v3/users/6c18cff85c8747f6902524ac29a284bb` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:46.647133 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234c51d5-47fa-4beb-869b-32beea2bfd9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c18cff85c8747f6902524ac29a284bb)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:46.652328 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234c51d5-47fa-4beb-869b-32beea2bfd9c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:46.676905 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:46.747619 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-234c51d5-47fa-4beb-869b-32beea2bfd9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c18cff85c8747f6902524ac29a284bb'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:46.748552 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-234c51d5-47fa-4beb-869b-32beea2bfd9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c18cff85c8747f6902524ac29a284bb'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:46.756794 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-33d7612a-872e-4074-90d3-87498ccced56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4637bd688ad74a31a739759076a523d3'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:46.759271 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-33d7612a-872e-4074-90d3-87498ccced56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4637bd688ad74a31a739759076a523d3'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:46.778509 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-234c51d5-47fa-4beb-869b-32beea2bfd9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c18cff85c8747f6902524ac29a284bb'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:46.779764 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-234c51d5-47fa-4beb-869b-32beea2bfd9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c18cff85c8747f6902524ac29a284bb'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:46.787347 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-33d7612a-872e-4074-90d3-87498ccced56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4637bd688ad74a31a739759076a523d3'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:46.791272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-234c51d5-47fa-4beb-869b-32beea2bfd9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c18cff85c8747f6902524ac29a284bb'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:46.798850 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-33d7612a-872e-4074-90d3-87498ccced56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4637bd688ad74a31a739759076a523d3'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:46.807520 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:46.811671 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-33d7612a-872e-4074-90d3-87498ccced56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4637bd688ad74a31a739759076a523d3'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:46.816690 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 935/1864] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:46 2022] DELETE /identity/v3/users/6c18cff85c8747f6902524ac29a284bb => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:46.819057 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803f660-1251-4110-b78e-128b2f8ac2d3 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:46.819216 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803f660-1251-4110-b78e-128b2f8ac2d3 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:46.819495 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803f660-1251-4110-b78e-128b2f8ac2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:46.820632 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 930/1865] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:46 2022] DELETE /identity/v3/users/4637bd688ad74a31a739759076a523d3 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:46.840738 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-8803f660-1251-4110-b78e-128b2f8ac2d3 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:46.843068 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-8803f660-1251-4110-b78e-128b2f8ac2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:46.920427 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 936/1866] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:46.937766 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d7612a-872e-4074-90d3-87498ccced56 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:46.939466 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-33d7612a-872e-4074-90d3-87498ccced56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:47.015102 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d7612a-872e-4074-90d3-87498ccced56 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:47.024630 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8d0a5-3292-4ba7-b39d-d1a325cac16f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:47.025413 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8d0a5-3292-4ba7-b39d-d1a325cac16f service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:47.025879 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8d0a5-3292-4ba7-b39d-d1a325cac16f service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:47.026728 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8d0a5-3292-4ba7-b39d-d1a325cac16f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:47.041526 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b8d0a5-3292-4ba7-b39d-d1a325cac16f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:47.043669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b8d0a5-3292-4ba7-b39d-d1a325cac16f service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:47.058868 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 931/1867] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:43:47.252013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8803f660-1251-4110-b78e-128b2f8ac2d3 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:47.253630 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-8803f660-1251-4110-b78e-128b2f8ac2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:47.270368 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8803f660-1251-4110-b78e-128b2f8ac2d3 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:47.289038 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dfe991-7430-46d7-96c1-01191e43c18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:47.290419 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dfe991-7430-46d7-96c1-01191e43c18d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:47.290729 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dfe991-7430-46d7-96c1-01191e43c18d admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:47.290975 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dfe991-7430-46d7-96c1-01191e43c18d admin admin] PATH_INFO: `/v3/projects/dff4c772dc2d42209a7771e2e1cc848c` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:47.292281 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dfe991-7430-46d7-96c1-01191e43c18d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dff4c772dc2d42209a7771e2e1cc848c)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:47.294944 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dfe991-7430-46d7-96c1-01191e43c18d admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:47.324634 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8d0a5-3292-4ba7-b39d-d1a325cac16f service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:47.326055 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d5b8d0a5-3292-4ba7-b39d-d1a325cac16f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:47.397749 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-69dfe991-7430-46d7-96c1-01191e43c18d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dff4c772dc2d42209a7771e2e1cc848c', 'request_id': 'req-69dfe991-7430-46d7-96c1-01191e43c18d'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:47.415584 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-69dfe991-7430-46d7-96c1-01191e43c18d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dff4c772dc2d42209a7771e2e1cc848c', 'request_id': 'req-69dfe991-7430-46d7-96c1-01191e43c18d'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:47.434772 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-69dfe991-7430-46d7-96c1-01191e43c18d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dff4c772dc2d42209a7771e2e1cc848c', 'request_id': 'req-69dfe991-7430-46d7-96c1-01191e43c18d'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:47.436495 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8d0a5-3292-4ba7-b39d-d1a325cac16f service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:47.452375 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-69dfe991-7430-46d7-96c1-01191e43c18d admin admin] The token cache is being invalidate because project dff4c772dc2d42209a7771e2e1cc848c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:47.452672 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-69dfe991-7430-46d7-96c1-01191e43c18d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:47.463503 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-69dfe991-7430-46d7-96c1-01191e43c18d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:47.465545 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 937/1868] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:47 2022] DELETE /identity/v3/projects/dff4c772dc2d42209a7771e2e1cc848c => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:47.470873 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7021296-6ddc-4d41-941b-1e55b2582dde None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:47.470873 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7021296-6ddc-4d41-941b-1e55b2582dde None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:47.470873 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7021296-6ddc-4d41-941b-1e55b2582dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:47.503971 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-f7021296-6ddc-4d41-941b-1e55b2582dde None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:47.507925 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f7021296-6ddc-4d41-941b-1e55b2582dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:47.522488 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:47.529442 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a15404-ed5e-4aad-90cc-405b361de239 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:47.530255 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a15404-ed5e-4aad-90cc-405b361de239 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:47.530706 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a15404-ed5e-4aad-90cc-405b361de239 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:47.531662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a15404-ed5e-4aad-90cc-405b361de239 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:47.535700 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f4a15404-ed5e-4aad-90cc-405b361de239 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:47.632099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 938/1869] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:47.641490 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852faf09-4636-4d38-9a45-0f394d86edb9 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:47.642043 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852faf09-4636-4d38-9a45-0f394d86edb9 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:47.642986 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852faf09-4636-4d38-9a45-0f394d86edb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:47.651959 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a15404-ed5e-4aad-90cc-405b361de239 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:47.653433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a15404-ed5e-4aad-90cc-405b361de239 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:47.668440 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 932/1870] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:47 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:43:47.684764 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-852faf09-4636-4d38-9a45-0f394d86edb9 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:47.686121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-852faf09-4636-4d38-9a45-0f394d86edb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:47.696107 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 939/1871] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:47.700550 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a15404-ed5e-4aad-90cc-405b361de239 service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:47.706249 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:47.719163 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a15404-ed5e-4aad-90cc-405b361de239 service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:47.738411 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b434a944-991a-4884-b2bf-7d2824e9e1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:47.739201 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434a944-991a-4884-b2bf-7d2824e9e1d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:47.739573 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434a944-991a-4884-b2bf-7d2824e9e1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:47.739893 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434a944-991a-4884-b2bf-7d2824e9e1d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:47.740744 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b434a944-991a-4884-b2bf-7d2824e9e1d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:43:47.741085 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b434a944-991a-4884-b2bf-7d2824e9e1d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:47.745100 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b434a944-991a-4884-b2bf-7d2824e9e1d2 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:47.777000 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 933/1872] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:43:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:43:47.779205 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852faf09-4636-4d38-9a45-0f394d86edb9 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:47.785865 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b434a944-991a-4884-b2bf-7d2824e9e1d2 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:47.788299 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b434a944-991a-4884-b2bf-7d2824e9e1d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:47.798214 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852faf09-4636-4d38-9a45-0f394d86edb9 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:47.810945 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de6d7b-f02a-4965-b0c6-18e6d26792f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:47.812174 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b434a944-991a-4884-b2bf-7d2824e9e1d2 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:47.812472 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de6d7b-f02a-4965-b0c6-18e6d26792f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:47.812875 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de6d7b-f02a-4965-b0c6-18e6d26792f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:47.816320 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de6d7b-f02a-4965-b0c6-18e6d26792f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:47.816320 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de6d7b-f02a-4965-b0c6-18e6d26792f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:47.816978 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de6d7b-f02a-4965-b0c6-18e6d26792f1 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:47.823755 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0606955-2319-454e-99ad-54ff3b52c276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:47.825633 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0606955-2319-454e-99ad-54ff3b52c276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:47.826492 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0606955-2319-454e-99ad-54ff3b52c276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:47.827067 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0606955-2319-454e-99ad-54ff3b52c276 admin admin] PATH_INFO: `/v3/projects/bfaa73656ee442a29e00013c1ce3b5d6` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:47.828761 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0606955-2319-454e-99ad-54ff3b52c276 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfaa73656ee442a29e00013c1ce3b5d6)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:47.831909 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0606955-2319-454e-99ad-54ff3b52c276 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:47.877185 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 940/1873] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:47 2022] POST /identity/v3/projects => generated 367 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:47.882618 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de6d7b-f02a-4965-b0c6-18e6d26792f1 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:47.962774 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a0606955-2319-454e-99ad-54ff3b52c276 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfaa73656ee442a29e00013c1ce3b5d6', 'request_id': 'req-a0606955-2319-454e-99ad-54ff3b52c276'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:47.981155 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a0606955-2319-454e-99ad-54ff3b52c276 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfaa73656ee442a29e00013c1ce3b5d6', 'request_id': 'req-a0606955-2319-454e-99ad-54ff3b52c276'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:48.008549 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a0606955-2319-454e-99ad-54ff3b52c276 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfaa73656ee442a29e00013c1ce3b5d6', 'request_id': 'req-a0606955-2319-454e-99ad-54ff3b52c276'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:48.033854 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a0606955-2319-454e-99ad-54ff3b52c276 admin admin] The token cache is being invalidate because project bfaa73656ee442a29e00013c1ce3b5d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:48.035245 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a0606955-2319-454e-99ad-54ff3b52c276 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:48.039310 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a0606955-2319-454e-99ad-54ff3b52c276 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:48.044666 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 934/1874] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:47 2022] DELETE /identity/v3/projects/bfaa73656ee442a29e00013c1ce3b5d6 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:43:48.045118 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de6d7b-f02a-4965-b0c6-18e6d26792f1 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:48.049755 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8af201-34e7-4ca8-a886-efe119119c90 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:48.050117 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8af201-34e7-4ca8-a886-efe119119c90 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:48.050501 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8af201-34e7-4ca8-a886-efe119119c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:48.076096 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-3e8af201-34e7-4ca8-a886-efe119119c90 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:48.077543 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-3e8af201-34e7-4ca8-a886-efe119119c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:48.185183 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099fc5a8-b4cc-4b12-8bb8-038105536c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:48.186786 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099fc5a8-b4cc-4b12-8bb8-038105536c46 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:48.187390 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099fc5a8-b4cc-4b12-8bb8-038105536c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:48.187675 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099fc5a8-b4cc-4b12-8bb8-038105536c46 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:48.188493 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099fc5a8-b4cc-4b12-8bb8-038105536c46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:48.194311 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099fc5a8-b4cc-4b12-8bb8-038105536c46 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:48.221357 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 935/1875] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:48.224262 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4545bf3-917f-425f-838a-78bd1d9073c6 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:48.224694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4545bf3-917f-425f-838a-78bd1d9073c6 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:48.225206 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4545bf3-917f-425f-838a-78bd1d9073c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:48.251066 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 941/1876] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:47 2022] POST /identity/v3/users => generated 284 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:48.258952 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099fc5a8-b4cc-4b12-8bb8-038105536c46 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:48.263914 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-099fc5a8-b4cc-4b12-8bb8-038105536c46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:48.282739 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-b4545bf3-917f-425f-838a-78bd1d9073c6 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:48.287900 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b4545bf3-917f-425f-838a-78bd1d9073c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:48.292078 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099fc5a8-b4cc-4b12-8bb8-038105536c46 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:48.307916 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 936/1877] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:48.308950 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4545bf3-917f-425f-838a-78bd1d9073c6 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:48.309368 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fdc86e-5419-40d1-bcad-0a6021e99018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:48.310125 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fdc86e-5419-40d1-bcad-0a6021e99018 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:48.311557 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fdc86e-5419-40d1-bcad-0a6021e99018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:48.312040 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fdc86e-5419-40d1-bcad-0a6021e99018 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:48.312896 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fdc86e-5419-40d1-bcad-0a6021e99018 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:48.316779 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fdc86e-5419-40d1-bcad-0a6021e99018 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:48.321240 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4545bf3-917f-425f-838a-78bd1d9073c6 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:48.326586 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8af5a2-cc54-4c3d-9b35-7538149b8bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:48.327425 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8af5a2-cc54-4c3d-9b35-7538149b8bab admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:48.327802 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8af5a2-cc54-4c3d-9b35-7538149b8bab admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:48.328329 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8af5a2-cc54-4c3d-9b35-7538149b8bab admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:48.329815 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8af5a2-cc54-4c3d-9b35-7538149b8bab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:43:48.330225 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8af5a2-cc54-4c3d-9b35-7538149b8bab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:48.333733 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8af5a2-cc54-4c3d-9b35-7538149b8bab admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:48.347902 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 942/1878] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:48 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:48.353677 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fdc86e-5419-40d1-bcad-0a6021e99018 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:48.359192 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 937/1879] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:43:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:43:48.365471 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8af5a2-cc54-4c3d-9b35-7538149b8bab admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:48.371883 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fdc86e-5419-40d1-bcad-0a6021e99018 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:48.388148 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8af5a2-cc54-4c3d-9b35-7538149b8bab admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:48.389185 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4ae58-ea77-42c8-864c-03980be8aa04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:48.389875 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4ae58-ea77-42c8-864c-03980be8aa04 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:48.390315 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4ae58-ea77-42c8-864c-03980be8aa04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:48.390616 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4ae58-ea77-42c8-864c-03980be8aa04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:48.391385 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe4ae58-ea77-42c8-864c-03980be8aa04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:48.394794 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe4ae58-ea77-42c8-864c-03980be8aa04 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:48.395895 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82613a-c777-4e20-bca2-b5a9f8fb47c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:48.397424 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82613a-c777-4e20-bca2-b5a9f8fb47c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:48.398634 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82613a-c777-4e20-bca2-b5a9f8fb47c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:48.399196 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82613a-c777-4e20-bca2-b5a9f8fb47c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:48.401919 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab82613a-c777-4e20-bca2-b5a9f8fb47c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:48.405173 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:48.406760 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab82613a-c777-4e20-bca2-b5a9f8fb47c1 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:48.438898 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 943/1880] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:48 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:48.439083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4ae58-ea77-42c8-864c-03980be8aa04 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:48.454454 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 938/1881] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:48 2022] POST /identity/v3/projects => generated 375 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:43:48.459542 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82613a-c777-4e20-bca2-b5a9f8fb47c1 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:48.477229 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4ae58-ea77-42c8-864c-03980be8aa04 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:48.514489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:48.555842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82613a-c777-4e20-bca2-b5a9f8fb47c1 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:48.561623 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ea6c1-732f-4d97-a62c-82cb5cffb719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:48.562674 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ea6c1-732f-4d97-a62c-82cb5cffb719 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:48.563121 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ea6c1-732f-4d97-a62c-82cb5cffb719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:48.563488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ea6c1-732f-4d97-a62c-82cb5cffb719 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:48.564289 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ea6c1-732f-4d97-a62c-82cb5cffb719 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:48.566295 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ea6c1-732f-4d97-a62c-82cb5cffb719 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:48.582902 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57f0ba-a2df-4bbb-bf70-68b444ac2f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:48.584708 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57f0ba-a2df-4bbb-bf70-68b444ac2f31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:48.584978 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57f0ba-a2df-4bbb-bf70-68b444ac2f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:48.585248 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57f0ba-a2df-4bbb-bf70-68b444ac2f31 admin admin] PATH_INFO: `/v3/projects/3f1aa0a39286423a9a7f27222c4fb61f/users/14429eb3f7924fa7bae3c869ebfc26f9/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:48.592297 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 939/1882] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:48 2022] POST /identity/v3/users => generated 288 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:48.595989 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ea6c1-732f-4d97-a62c-82cb5cffb719 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:48.602449 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da57f0ba-a2df-4bbb-bf70-68b444ac2f31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f1aa0a39286423a9a7f27222c4fb61f, user_id=14429eb3f7924fa7bae3c869ebfc26f9, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:48.605412 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da57f0ba-a2df-4bbb-bf70-68b444ac2f31 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:48.608841 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ea6c1-732f-4d97-a62c-82cb5cffb719 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:48.614457 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8290c-056d-4b12-9853-2ed1a6853271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:48.615174 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8290c-056d-4b12-9853-2ed1a6853271 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:48.615461 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8290c-056d-4b12-9853-2ed1a6853271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:48.615972 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8290c-056d-4b12-9853-2ed1a6853271 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:48.616459 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca8290c-056d-4b12-9853-2ed1a6853271 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:48.618058 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca8290c-056d-4b12-9853-2ed1a6853271 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:48.621363 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 944/1883] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:48 2022] PUT /identity/v3/projects/3f1aa0a39286423a9a7f27222c4fb61f/users/14429eb3f7924fa7bae3c869ebfc26f9/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:43:48.630842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 940/1884] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:48.633769 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a195f-5c15-49bd-879d-615d1a55e9b6 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:48.633769 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a195f-5c15-49bd-879d-615d1a55e9b6 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:48.633769 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a195f-5c15-49bd-879d-615d1a55e9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:48.635748 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8290c-056d-4b12-9853-2ed1a6853271 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:48.648145 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8290c-056d-4b12-9853-2ed1a6853271 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:48.663344 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-df0a195f-5c15-49bd-879d-615d1a55e9b6 None None] MFA Rules not processed for user `14429eb3f7924fa7bae3c869ebfc26f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:48.665220 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-df0a195f-5c15-49bd-879d-615d1a55e9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:48.728991 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1135782e-d7ab-4a13-8eff-5f102961c049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:48.729796 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135782e-d7ab-4a13-8eff-5f102961c049 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:48.730157 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135782e-d7ab-4a13-8eff-5f102961c049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:48.730445 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135782e-d7ab-4a13-8eff-5f102961c049 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:48.731408 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1135782e-d7ab-4a13-8eff-5f102961c049 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:48.733037 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1135782e-d7ab-4a13-8eff-5f102961c049 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:48.744766 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 941/1885] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:48 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:48.748779 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1135782e-d7ab-4a13-8eff-5f102961c049 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:48.764049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1135782e-d7ab-4a13-8eff-5f102961c049 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:48.777777 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d52564-f10c-4d65-8f2d-47c6a303a444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:48.778701 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d52564-f10c-4d65-8f2d-47c6a303a444 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:48.779012 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d52564-f10c-4d65-8f2d-47c6a303a444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:48.779293 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d52564-f10c-4d65-8f2d-47c6a303a444 admin admin] PATH_INFO: `/v3/projects/73bd4e75069d41d3bb9ebfb932814f08/users/e89c9a7f88b04fdbac6a3927241dd4ce/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:48.790562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 945/1886] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:48 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:48.795302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a195f-5c15-49bd-879d-615d1a55e9b6 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:48.808975 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a195f-5c15-49bd-879d-615d1a55e9b6 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:48.815083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8601c-762d-4d0c-b713-f2723f5c165d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:48.817181 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8601c-762d-4d0c-b713-f2723f5c165d admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:48.817181 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8601c-762d-4d0c-b713-f2723f5c165d admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:48.817181 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8601c-762d-4d0c-b713-f2723f5c165d admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:48.818598 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba8601c-762d-4d0c-b713-f2723f5c165d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:48.820541 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba8601c-762d-4d0c-b713-f2723f5c165d admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:48.820658 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d52564-f10c-4d65-8f2d-47c6a303a444 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73bd4e75069d41d3bb9ebfb932814f08, user_id=e89c9a7f88b04fdbac6a3927241dd4ce, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:48.823233 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d52564-f10c-4d65-8f2d-47c6a303a444 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:48.848296 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 942/1887] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:48 2022] PUT /identity/v3/projects/73bd4e75069d41d3bb9ebfb932814f08/users/e89c9a7f88b04fdbac6a3927241dd4ce/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:48.858732 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb954464-67ff-4477-846a-71556c12dc29 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:48.859245 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb954464-67ff-4477-846a-71556c12dc29 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:48.859592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb954464-67ff-4477-846a-71556c12dc29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:48.861142 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 946/1888] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:48 2022] POST /identity/v3/projects => generated 369 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:48.866534 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8601c-762d-4d0c-b713-f2723f5c165d admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:48.900722 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-bb954464-67ff-4477-846a-71556c12dc29 None None] MFA Rules not processed for user `e89c9a7f88b04fdbac6a3927241dd4ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:48.902540 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-bb954464-67ff-4477-846a-71556c12dc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:48.951943 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:48.968322 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8601c-762d-4d0c-b713-f2723f5c165d admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:48.973754 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc34ce-17fb-43f3-8a0c-5d8fcdc05aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:48.974411 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc34ce-17fb-43f3-8a0c-5d8fcdc05aca admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:48.974705 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc34ce-17fb-43f3-8a0c-5d8fcdc05aca admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:48.975011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc34ce-17fb-43f3-8a0c-5d8fcdc05aca admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:48.975734 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcc34ce-17fb-43f3-8a0c-5d8fcdc05aca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:48.978300 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcc34ce-17fb-43f3-8a0c-5d8fcdc05aca admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:49.005699 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 943/1889] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:43:49.006518 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 947/1890] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:48 2022] POST /identity/v3/users => generated 291 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:49.010333 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb954464-67ff-4477-846a-71556c12dc29 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:49.022187 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc34ce-17fb-43f3-8a0c-5d8fcdc05aca admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:49.023776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb954464-67ff-4477-846a-71556c12dc29 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:49.024572 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-9bcc34ce-17fb-43f3-8a0c-5d8fcdc05aca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:49.030803 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d65445-ce34-45b8-9532-753883d8a418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:49.031731 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d65445-ce34-45b8-9532-753883d8a418 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:49.032083 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d65445-ce34-45b8-9532-753883d8a418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:49.032410 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d65445-ce34-45b8-9532-753883d8a418 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:49.033889 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d65445-ce34-45b8-9532-753883d8a418 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:49.039593 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d65445-ce34-45b8-9532-753883d8a418 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:49.047870 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:49.057962 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 944/1891] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:49 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:43:49.063373 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d65445-ce34-45b8-9532-753883d8a418 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:49.079863 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d65445-ce34-45b8-9532-753883d8a418 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:49.103621 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6572c854-64fa-4a4f-b4ed-54e58b82df47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:49.104718 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572c854-64fa-4a4f-b4ed-54e58b82df47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:49.105011 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572c854-64fa-4a4f-b4ed-54e58b82df47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:49.106451 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572c854-64fa-4a4f-b4ed-54e58b82df47 admin admin] PATH_INFO: `/v3/projects/f0e4b7441e2a4aaa9c85f040112cc792/users/058d38db28ea42dfa3f91d09319affc4/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:49.133491 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6572c854-64fa-4a4f-b4ed-54e58b82df47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0e4b7441e2a4aaa9c85f040112cc792, user_id=058d38db28ea42dfa3f91d09319affc4, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:49.137935 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6572c854-64fa-4a4f-b4ed-54e58b82df47 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:49.140708 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc34ce-17fb-43f3-8a0c-5d8fcdc05aca admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:49.145222 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f8343-e1fe-4353-a1ef-10f8a9ef705b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:49.146404 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f8343-e1fe-4353-a1ef-10f8a9ef705b service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:49.146688 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f8343-e1fe-4353-a1ef-10f8a9ef705b service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:49.146974 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f8343-e1fe-4353-a1ef-10f8a9ef705b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:49.156734 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547f8343-e1fe-4353-a1ef-10f8a9ef705b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:49.157900 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 945/1892] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:49 2022] PUT /identity/v3/projects/f0e4b7441e2a4aaa9c85f040112cc792/users/058d38db28ea42dfa3f91d09319affc4/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:49.158136 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547f8343-e1fe-4353-a1ef-10f8a9ef705b service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:49.158845 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6572c854-64fa-4a4f-b4ed-54e58b82df47 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:49.170982 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 948/1893] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:43:49.257204 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6572c854-64fa-4a4f-b4ed-54e58b82df47 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:49.263669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a8559-e7b5-47d4-845a-34aa5f8d2a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:49.264595 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a8559-e7b5-47d4-845a-34aa5f8d2a70 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:49.265055 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a8559-e7b5-47d4-845a-34aa5f8d2a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:49.265757 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a8559-e7b5-47d4-845a-34aa5f8d2a70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:49.267640 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766a8559-e7b5-47d4-845a-34aa5f8d2a70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:49.270654 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766a8559-e7b5-47d4-845a-34aa5f8d2a70 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:49.297208 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 946/1894] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:49 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:49.298836 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f8343-e1fe-4353-a1ef-10f8a9ef705b service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:49.319105 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f8343-e1fe-4353-a1ef-10f8a9ef705b service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:49.332960 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c95725-e989-44ae-a9d0-64937755d1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:49.333964 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c95725-e989-44ae-a9d0-64937755d1d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:49.334666 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c95725-e989-44ae-a9d0-64937755d1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:49.335098 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c95725-e989-44ae-a9d0-64937755d1d5 admin admin] PATH_INFO: `/v3/projects/f0e4b7441e2a4aaa9c85f040112cc792/users/058d38db28ea42dfa3f91d09319affc4/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:49.341219 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c95725-e989-44ae-a9d0-64937755d1d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0e4b7441e2a4aaa9c85f040112cc792, user_id=058d38db28ea42dfa3f91d09319affc4, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:49.345778 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c95725-e989-44ae-a9d0-64937755d1d5 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:49.371354 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 949/1895] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:49 2022] PUT /identity/v3/projects/f0e4b7441e2a4aaa9c85f040112cc792/users/058d38db28ea42dfa3f91d09319affc4/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:49.388215 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce990ee-aefe-4b4c-b2ae-5f69620d3dc2 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:49.388807 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce990ee-aefe-4b4c-b2ae-5f69620d3dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:49.389152 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce990ee-aefe-4b4c-b2ae-5f69620d3dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:49.425068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-cce990ee-aefe-4b4c-b2ae-5f69620d3dc2 None None] MFA Rules not processed for user `058d38db28ea42dfa3f91d09319affc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:49.427543 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-cce990ee-aefe-4b4c-b2ae-5f69620d3dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:49.550873 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 947/1896] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:49 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:49.574109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c95725-e989-44ae-a9d0-64937755d1d5 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:49.653366 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c95725-e989-44ae-a9d0-64937755d1d5 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:49.660808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38efb26-5e63-4791-af30-55878d9b6d1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:49.661529 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38efb26-5e63-4791-af30-55878d9b6d1d service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:49.661843 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38efb26-5e63-4791-af30-55878d9b6d1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:49.662176 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38efb26-5e63-4791-af30-55878d9b6d1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:49.672661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38efb26-5e63-4791-af30-55878d9b6d1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:49.674167 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38efb26-5e63-4791-af30-55878d9b6d1d service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:49.685931 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 950/1897] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:49 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:49.736737 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce990ee-aefe-4b4c-b2ae-5f69620d3dc2 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:49.738324 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-cce990ee-aefe-4b4c-b2ae-5f69620d3dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:49.752333 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce990ee-aefe-4b4c-b2ae-5f69620d3dc2 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:49.756099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629bde5f-9fee-42c8-ac84-6afb38a33806 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:49.756766 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bde5f-9fee-42c8-ac84-6afb38a33806 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:49.757050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bde5f-9fee-42c8-ac84-6afb38a33806 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:49.757327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bde5f-9fee-42c8-ac84-6afb38a33806 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:49.823427 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:49.829312 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629bde5f-9fee-42c8-ac84-6afb38a33806 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:49.830668 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629bde5f-9fee-42c8-ac84-6afb38a33806 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:49.848250 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 948/1898] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:49 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:53.453207 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38efb26-5e63-4791-af30-55878d9b6d1d service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:53.519467 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38efb26-5e63-4791-af30-55878d9b6d1d service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:53.524932 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec19415b-2ce9-4aeb-9b26-70c294ff05ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:53.525787 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19415b-2ce9-4aeb-9b26-70c294ff05ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:53.526003 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19415b-2ce9-4aeb-9b26-70c294ff05ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:53.526521 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19415b-2ce9-4aeb-9b26-70c294ff05ff admin admin] PATH_INFO: `/v3/users/14429eb3f7924fa7bae3c869ebfc26f9` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:53.527812 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec19415b-2ce9-4aeb-9b26-70c294ff05ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=14429eb3f7924fa7bae3c869ebfc26f9)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:53.529746 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec19415b-2ce9-4aeb-9b26-70c294ff05ff admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:53.543621 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:53.583976 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ec19415b-2ce9-4aeb-9b26-70c294ff05ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14429eb3f7924fa7bae3c869ebfc26f9'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:53.584645 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ec19415b-2ce9-4aeb-9b26-70c294ff05ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14429eb3f7924fa7bae3c869ebfc26f9'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:53.597822 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ec19415b-2ce9-4aeb-9b26-70c294ff05ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14429eb3f7924fa7bae3c869ebfc26f9'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:53.598173 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ec19415b-2ce9-4aeb-9b26-70c294ff05ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14429eb3f7924fa7bae3c869ebfc26f9'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:53.609007 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ec19415b-2ce9-4aeb-9b26-70c294ff05ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14429eb3f7924fa7bae3c869ebfc26f9'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:53.622382 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 951/1899] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:53 2022] DELETE /identity/v3/users/14429eb3f7924fa7bae3c869ebfc26f9 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:53.627107 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2027e-117a-4691-8081-298edb2c40ac None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:53.627584 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2027e-117a-4691-8081-298edb2c40ac None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:53.627676 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2027e-117a-4691-8081-298edb2c40ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:53.645017 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-96c2027e-117a-4691-8081-298edb2c40ac None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:53.646755 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-96c2027e-117a-4691-8081-298edb2c40ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:53.729614 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 949/1900] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:53.739511 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec19415b-2ce9-4aeb-9b26-70c294ff05ff admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:53.741016 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ec19415b-2ce9-4aeb-9b26-70c294ff05ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:53.824399 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec19415b-2ce9-4aeb-9b26-70c294ff05ff admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:53.829405 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58363e93-208d-4589-aee5-f7a93493331c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:53.830060 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58363e93-208d-4589-aee5-f7a93493331c service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:53.830363 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58363e93-208d-4589-aee5-f7a93493331c service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:53.830624 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58363e93-208d-4589-aee5-f7a93493331c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:53.841855 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58363e93-208d-4589-aee5-f7a93493331c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:53.843392 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58363e93-208d-4589-aee5-f7a93493331c service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:53.854118 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 952/1901] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:54.184005 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2027e-117a-4691-8081-298edb2c40ac None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:54.185510 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-96c2027e-117a-4691-8081-298edb2c40ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:54.197937 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2027e-117a-4691-8081-298edb2c40ac None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:54.203409 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14841a0-4069-48a2-b345-56664c41c96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:54.204502 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14841a0-4069-48a2-b345-56664c41c96c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:54.204784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14841a0-4069-48a2-b345-56664c41c96c admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:54.205067 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14841a0-4069-48a2-b345-56664c41c96c admin admin] PATH_INFO: `/v3/projects/3f1aa0a39286423a9a7f27222c4fb61f` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:54.206818 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14841a0-4069-48a2-b345-56664c41c96c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f1aa0a39286423a9a7f27222c4fb61f)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:54.209919 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14841a0-4069-48a2-b345-56664c41c96c admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:54.262776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b14841a0-4069-48a2-b345-56664c41c96c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f1aa0a39286423a9a7f27222c4fb61f', 'request_id': 'req-b14841a0-4069-48a2-b345-56664c41c96c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:54.272803 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b14841a0-4069-48a2-b345-56664c41c96c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f1aa0a39286423a9a7f27222c4fb61f', 'request_id': 'req-b14841a0-4069-48a2-b345-56664c41c96c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:54.279535 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b14841a0-4069-48a2-b345-56664c41c96c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f1aa0a39286423a9a7f27222c4fb61f', 'request_id': 'req-b14841a0-4069-48a2-b345-56664c41c96c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:54.291173 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b14841a0-4069-48a2-b345-56664c41c96c admin admin] The token cache is being invalidate because project 3f1aa0a39286423a9a7f27222c4fb61f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:54.291464 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b14841a0-4069-48a2-b345-56664c41c96c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:54.292124 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b14841a0-4069-48a2-b345-56664c41c96c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:54.293735 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 950/1902] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:54 2022] DELETE /identity/v3/projects/3f1aa0a39286423a9a7f27222c4fb61f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:54.297671 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58363e93-208d-4589-aee5-f7a93493331c service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:54.307581 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58363e93-208d-4589-aee5-f7a93493331c service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:54.389474 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f72aa7-5c72-42ed-9dd6-33242c106b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:54.390493 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f72aa7-5c72-42ed-9dd6-33242c106b9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:54.390781 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f72aa7-5c72-42ed-9dd6-33242c106b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:54.391090 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f72aa7-5c72-42ed-9dd6-33242c106b9f admin admin] PATH_INFO: `/v3/users/058d38db28ea42dfa3f91d09319affc4` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:54.392283 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f72aa7-5c72-42ed-9dd6-33242c106b9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=058d38db28ea42dfa3f91d09319affc4)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:54.394245 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f72aa7-5c72-42ed-9dd6-33242c106b9f admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:54.438040 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:54.453207 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-57f72aa7-5c72-42ed-9dd6-33242c106b9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '058d38db28ea42dfa3f91d09319affc4'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:54.454422 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-57f72aa7-5c72-42ed-9dd6-33242c106b9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '058d38db28ea42dfa3f91d09319affc4'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:54.475105 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-57f72aa7-5c72-42ed-9dd6-33242c106b9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '058d38db28ea42dfa3f91d09319affc4'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:54.475989 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-57f72aa7-5c72-42ed-9dd6-33242c106b9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '058d38db28ea42dfa3f91d09319affc4'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:54.484917 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-57f72aa7-5c72-42ed-9dd6-33242c106b9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '058d38db28ea42dfa3f91d09319affc4'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:54.500311 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 953/1903] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:54 2022] DELETE /identity/v3/users/058d38db28ea42dfa3f91d09319affc4 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:55.196809 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14841a0-4069-48a2-b345-56664c41c96c admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:55.198802 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b14841a0-4069-48a2-b345-56664c41c96c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:55.217858 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14841a0-4069-48a2-b345-56664c41c96c admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:55.309052 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6c2c9-7d2f-45ad-a457-e97b2a3ed867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:55.310953 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6c2c9-7d2f-45ad-a457-e97b2a3ed867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:55.311331 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6c2c9-7d2f-45ad-a457-e97b2a3ed867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:55.311638 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6c2c9-7d2f-45ad-a457-e97b2a3ed867 admin admin] PATH_INFO: `/v3/projects/f0e4b7441e2a4aaa9c85f040112cc792` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:55.315265 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d6c2c9-7d2f-45ad-a457-e97b2a3ed867 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0e4b7441e2a4aaa9c85f040112cc792)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:55.315482 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d6c2c9-7d2f-45ad-a457-e97b2a3ed867 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:55.346700 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:55.403651 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-90d6c2c9-7d2f-45ad-a457-e97b2a3ed867 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0e4b7441e2a4aaa9c85f040112cc792', 'request_id': 'req-90d6c2c9-7d2f-45ad-a457-e97b2a3ed867'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:55.426139 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-90d6c2c9-7d2f-45ad-a457-e97b2a3ed867 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0e4b7441e2a4aaa9c85f040112cc792', 'request_id': 'req-90d6c2c9-7d2f-45ad-a457-e97b2a3ed867'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:55.436597 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-90d6c2c9-7d2f-45ad-a457-e97b2a3ed867 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0e4b7441e2a4aaa9c85f040112cc792', 'request_id': 'req-90d6c2c9-7d2f-45ad-a457-e97b2a3ed867'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:55.455542 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-90d6c2c9-7d2f-45ad-a457-e97b2a3ed867 admin admin] The token cache is being invalidate because project f0e4b7441e2a4aaa9c85f040112cc792 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:55.455872 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-90d6c2c9-7d2f-45ad-a457-e97b2a3ed867 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:55.456561 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-90d6c2c9-7d2f-45ad-a457-e97b2a3ed867 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:55.459502 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 951/1904] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:55 2022] DELETE /identity/v3/projects/f0e4b7441e2a4aaa9c85f040112cc792 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:55.464077 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c62c6-8ddb-47a0-852a-e5acbe80087a None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:55.464544 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c62c6-8ddb-47a0-852a-e5acbe80087a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:55.464723 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c62c6-8ddb-47a0-852a-e5acbe80087a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:55.484440 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-476c62c6-8ddb-47a0-852a-e5acbe80087a None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:55.485680 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-476c62c6-8ddb-47a0-852a-e5acbe80087a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:55.570806 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 954/1905] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:55.578809 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a8763-de12-41f6-9958-970ac4f93ead None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:55.579147 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a8763-de12-41f6-9958-970ac4f93ead None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:55.579428 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a8763-de12-41f6-9958-970ac4f93ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:55.602256 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-1d5a8763-de12-41f6-9958-970ac4f93ead None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:55.603642 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1d5a8763-de12-41f6-9958-970ac4f93ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:55.613445 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 952/1906] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:55.618299 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c62c6-8ddb-47a0-852a-e5acbe80087a None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:55.630166 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c62c6-8ddb-47a0-852a-e5acbe80087a None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:55.635836 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c15d43-2fd6-48cc-9c71-a45b5aa404a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:55.636510 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c15d43-2fd6-48cc-9c71-a45b5aa404a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:55.636795 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c15d43-2fd6-48cc-9c71-a45b5aa404a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:55.637075 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c15d43-2fd6-48cc-9c71-a45b5aa404a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:55.637805 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c15d43-2fd6-48cc-9c71-a45b5aa404a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:43:55.638114 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c15d43-2fd6-48cc-9c71-a45b5aa404a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:55.639768 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c15d43-2fd6-48cc-9c71-a45b5aa404a6 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:55.655029 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 955/1907] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:43:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:43:55.657818 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a8763-de12-41f6-9958-970ac4f93ead None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:55.671956 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a8763-de12-41f6-9958-970ac4f93ead None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:55.680443 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b0161-08c1-41fa-a964-00b3b3a96dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:55.681722 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b0161-08c1-41fa-a964-00b3b3a96dc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:55.681722 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b0161-08c1-41fa-a964-00b3b3a96dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:55.681913 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b0161-08c1-41fa-a964-00b3b3a96dc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:55.682491 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b0161-08c1-41fa-a964-00b3b3a96dc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:55.685385 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b0161-08c1-41fa-a964-00b3b3a96dc7 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:55.719714 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 953/1908] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:55 2022] POST /identity/v3/projects => generated 381 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:55.725218 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c15d43-2fd6-48cc-9c71-a45b5aa404a6 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:55.738826 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c15d43-2fd6-48cc-9c71-a45b5aa404a6 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:55.816711 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c50a4e-e047-46db-afd6-2033916709e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:55.817466 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c50a4e-e047-46db-afd6-2033916709e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:55.817765 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c50a4e-e047-46db-afd6-2033916709e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:55.818538 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c50a4e-e047-46db-afd6-2033916709e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:55.819303 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c50a4e-e047-46db-afd6-2033916709e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:55.821332 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c50a4e-e047-46db-afd6-2033916709e9 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:55.843566 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 956/1909] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:55 2022] POST /identity/v3/users => generated 291 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:55.848177 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b0161-08c1-41fa-a964-00b3b3a96dc7 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:55.862929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b0161-08c1-41fa-a964-00b3b3a96dc7 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:55.869437 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f1bfd-0afe-43fa-ad79-95019dd1e501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:55.870212 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f1bfd-0afe-43fa-ad79-95019dd1e501 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:55.870696 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f1bfd-0afe-43fa-ad79-95019dd1e501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:55.871397 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f1bfd-0afe-43fa-ad79-95019dd1e501 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:55.871934 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f1bfd-0afe-43fa-ad79-95019dd1e501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:55.874229 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f1bfd-0afe-43fa-ad79-95019dd1e501 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:55.887786 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 954/1910] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:55.891965 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c50a4e-e047-46db-afd6-2033916709e9 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:55.903140 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c50a4e-e047-46db-afd6-2033916709e9 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:55.908018 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e596019-0adb-4e7f-b968-ec9fa8f4200b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:55.908697 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e596019-0adb-4e7f-b968-ec9fa8f4200b admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:55.908981 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e596019-0adb-4e7f-b968-ec9fa8f4200b admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:55.909254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e596019-0adb-4e7f-b968-ec9fa8f4200b admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:55.909996 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e596019-0adb-4e7f-b968-ec9fa8f4200b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:55.912266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e596019-0adb-4e7f-b968-ec9fa8f4200b admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:55.923060 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 957/1911] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:55.927450 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f1bfd-0afe-43fa-ad79-95019dd1e501 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:55.939092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f1bfd-0afe-43fa-ad79-95019dd1e501 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:55.943745 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ca653-45eb-4d32-9ac9-be291cb5afcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:55.944630 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ca653-45eb-4d32-9ac9-be291cb5afcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:55.944914 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ca653-45eb-4d32-9ac9-be291cb5afcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:55.945185 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ca653-45eb-4d32-9ac9-be291cb5afcf admin admin] PATH_INFO: `/v3/projects/924f76839cdf4e63bd0377990a4a0f0a/users/4967ed0bde3f42e0a9119b54d99db1eb/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:55.961862 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611ca653-45eb-4d32-9ac9-be291cb5afcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=924f76839cdf4e63bd0377990a4a0f0a, user_id=4967ed0bde3f42e0a9119b54d99db1eb, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:55.964637 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611ca653-45eb-4d32-9ac9-be291cb5afcf admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:55.978450 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 955/1912] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:55 2022] PUT /identity/v3/projects/924f76839cdf4e63bd0377990a4a0f0a/users/4967ed0bde3f42e0a9119b54d99db1eb/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:55.988842 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80985ff2-742a-4b63-9948-52d1e23a2c1b None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:55.989129 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80985ff2-742a-4b63-9948-52d1e23a2c1b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:55.989457 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80985ff2-742a-4b63-9948-52d1e23a2c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:56.016670 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-80985ff2-742a-4b63-9948-52d1e23a2c1b None None] MFA Rules not processed for user `4967ed0bde3f42e0a9119b54d99db1eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:56.017901 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-80985ff2-742a-4b63-9948-52d1e23a2c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:56.025469 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:56.091520 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 958/1913] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:56.096031 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ca653-45eb-4d32-9ac9-be291cb5afcf admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:56.163965 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ca653-45eb-4d32-9ac9-be291cb5afcf admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:56.168765 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6f0ef-c08c-443f-9bae-ab4f7e62d09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:56.169460 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6f0ef-c08c-443f-9bae-ab4f7e62d09f admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:56.169791 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6f0ef-c08c-443f-9bae-ab4f7e62d09f admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:56.170105 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6f0ef-c08c-443f-9bae-ab4f7e62d09f admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:56.170943 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce6f0ef-c08c-443f-9bae-ab4f7e62d09f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:56.173021 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce6f0ef-c08c-443f-9bae-ab4f7e62d09f admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:56.199971 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 956/1914] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:56 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:43:56.204136 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80985ff2-742a-4b63-9948-52d1e23a2c1b None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:56.215043 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80985ff2-742a-4b63-9948-52d1e23a2c1b None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:56.273376 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d7715-3c1b-4349-96b0-fdbc0d3c9dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:56.274059 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d7715-3c1b-4349-96b0-fdbc0d3c9dac admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:56.274342 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d7715-3c1b-4349-96b0-fdbc0d3c9dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:56.274614 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d7715-3c1b-4349-96b0-fdbc0d3c9dac admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:56.275352 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06d7715-3c1b-4349-96b0-fdbc0d3c9dac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:56.277561 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06d7715-3c1b-4349-96b0-fdbc0d3c9dac admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:56.300360 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 959/1915] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:56 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:43:56.305115 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6f0ef-c08c-443f-9bae-ab4f7e62d09f admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:56.315511 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6f0ef-c08c-443f-9bae-ab4f7e62d09f admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:56.321064 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad325fb-9e09-4c14-bdac-b0cb40424695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:56.321743 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad325fb-9e09-4c14-bdac-b0cb40424695 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:56.322035 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad325fb-9e09-4c14-bdac-b0cb40424695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:56.322308 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad325fb-9e09-4c14-bdac-b0cb40424695 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:56.323079 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad325fb-9e09-4c14-bdac-b0cb40424695 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:56.326198 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad325fb-9e09-4c14-bdac-b0cb40424695 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:56.341308 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 957/1916] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:43:56.345929 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d7715-3c1b-4349-96b0-fdbc0d3c9dac admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:56.357249 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d7715-3c1b-4349-96b0-fdbc0d3c9dac admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:56.368425 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2765cb13-bb94-44e7-bccb-3d225f74f7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:56.370409 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765cb13-bb94-44e7-bccb-3d225f74f7cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:56.370937 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765cb13-bb94-44e7-bccb-3d225f74f7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:56.371342 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765cb13-bb94-44e7-bccb-3d225f74f7cf admin admin] PATH_INFO: `/v3/projects/19898f1737114fb7ac338122150aa72e/users/6f0c64ced070444e832d9acfdea70939/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:56.389069 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2765cb13-bb94-44e7-bccb-3d225f74f7cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=19898f1737114fb7ac338122150aa72e, user_id=6f0c64ced070444e832d9acfdea70939, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:56.391829 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2765cb13-bb94-44e7-bccb-3d225f74f7cf admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:56.416203 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 960/1917] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:56 2022] PUT /identity/v3/projects/19898f1737114fb7ac338122150aa72e/users/6f0c64ced070444e832d9acfdea70939/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:56.417568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad325fb-9e09-4c14-bdac-b0cb40424695 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:56.428645 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad325fb-9e09-4c14-bdac-b0cb40424695 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:56.452904 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:56.504782 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032513f1-81eb-4747-b69c-9a7de9abbd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:56.505486 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032513f1-81eb-4747-b69c-9a7de9abbd8c admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:56.505784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032513f1-81eb-4747-b69c-9a7de9abbd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:56.506065 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032513f1-81eb-4747-b69c-9a7de9abbd8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:56.506798 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032513f1-81eb-4747-b69c-9a7de9abbd8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:56.508556 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032513f1-81eb-4747-b69c-9a7de9abbd8c admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:56.519474 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 958/1918] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:43:56.523724 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2765cb13-bb94-44e7-bccb-3d225f74f7cf admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:56.537124 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2765cb13-bb94-44e7-bccb-3d225f74f7cf admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:56.541935 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85787a7-f2fa-4e5d-900a-d7ab1c125970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:56.542830 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85787a7-f2fa-4e5d-900a-d7ab1c125970 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:56.543140 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85787a7-f2fa-4e5d-900a-d7ab1c125970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:56.543422 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85787a7-f2fa-4e5d-900a-d7ab1c125970 admin admin] PATH_INFO: `/v3/projects/19898f1737114fb7ac338122150aa72e/users/6f0c64ced070444e832d9acfdea70939/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:56.545408 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85787a7-f2fa-4e5d-900a-d7ab1c125970 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19898f1737114fb7ac338122150aa72e, user_id=6f0c64ced070444e832d9acfdea70939, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:56.548068 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85787a7-f2fa-4e5d-900a-d7ab1c125970 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:56.558780 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 961/1919] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:56 2022] PUT /identity/v3/projects/19898f1737114fb7ac338122150aa72e/users/6f0c64ced070444e832d9acfdea70939/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:56.570077 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f11dc-2c23-4e4e-9101-03cf43cb1245 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:56.570366 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f11dc-2c23-4e4e-9101-03cf43cb1245 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:56.570645 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f11dc-2c23-4e4e-9101-03cf43cb1245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:56.598360 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-fc4f11dc-2c23-4e4e-9101-03cf43cb1245 None None] MFA Rules not processed for user `6f0c64ced070444e832d9acfdea70939`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:56.600349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-fc4f11dc-2c23-4e4e-9101-03cf43cb1245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:56.668401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 959/1920] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:56 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:56.681006 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85787a7-f2fa-4e5d-900a-d7ab1c125970 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:56.682719 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c85787a7-f2fa-4e5d-900a-d7ab1c125970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:56.748761 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85787a7-f2fa-4e5d-900a-d7ab1c125970 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:56.752587 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f0e33-4bce-4902-9679-ef20e0ea54ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:56.753305 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f0e33-4bce-4902-9679-ef20e0ea54ad service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:56.753678 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f0e33-4bce-4902-9679-ef20e0ea54ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:56.754019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f0e33-4bce-4902-9679-ef20e0ea54ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:56.763719 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311f0e33-4bce-4902-9679-ef20e0ea54ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:56.765268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311f0e33-4bce-4902-9679-ef20e0ea54ad service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:56.775979 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 962/1921] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:56 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:57.132740 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f11dc-2c23-4e4e-9101-03cf43cb1245 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:57.142246 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f11dc-2c23-4e4e-9101-03cf43cb1245 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:57.152885 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1fad4-6cc4-4348-8021-f8f2faa72e40 tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0c64ced070444e832d9acfdea70939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19898f1737114fb7ac338122150aa72e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:57.154346 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1fad4-6cc4-4348-8021-f8f2faa72e40 tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:57.154644 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1fad4-6cc4-4348-8021-f8f2faa72e40 tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:57.154938 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1fad4-6cc4-4348-8021-f8f2faa72e40 tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:57.155724 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d1fad4-6cc4-4348-8021-f8f2faa72e40 tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:43:57.156045 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d1fad4-6cc4-4348-8021-f8f2faa72e40 tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:57.157696 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d1fad4-6cc4-4348-8021-f8f2faa72e40 tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:57.175044 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 960/1922] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:43:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:43:57.180295 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f0e33-4bce-4902-9679-ef20e0ea54ad service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:57.184619 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:57.192058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f0e33-4bce-4902-9679-ef20e0ea54ad service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:57.197475 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123f8d0a-fece-4485-b301-5aa495237518 tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0c64ced070444e832d9acfdea70939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19898f1737114fb7ac338122150aa72e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:57.198399 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f8d0a-fece-4485-b301-5aa495237518 tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:57.198789 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f8d0a-fece-4485-b301-5aa495237518 tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:57.199214 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f8d0a-fece-4485-b301-5aa495237518 tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:57.200014 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123f8d0a-fece-4485-b301-5aa495237518 tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:57.207100 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123f8d0a-fece-4485-b301-5aa495237518 tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:57.231412 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 963/1923] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:57 2022] POST /identity/v3/projects => generated 354 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:57.235717 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1fad4-6cc4-4348-8021-f8f2faa72e40 tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:57.246216 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1fad4-6cc4-4348-8021-f8f2faa72e40 tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:57.305844 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79c867-0b72-4bb0-b507-ce68c40df2af tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0c64ced070444e832d9acfdea70939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19898f1737114fb7ac338122150aa72e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:57.306517 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79c867-0b72-4bb0-b507-ce68c40df2af tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:57.306804 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79c867-0b72-4bb0-b507-ce68c40df2af tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:57.307111 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79c867-0b72-4bb0-b507-ce68c40df2af tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:57.307842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79c867-0b72-4bb0-b507-ce68c40df2af tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:43:57.308154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79c867-0b72-4bb0-b507-ce68c40df2af tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:57.309750 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79c867-0b72-4bb0-b507-ce68c40df2af tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:57.321894 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 961/1924] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:43:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:43:57.367231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123f8d0a-fece-4485-b301-5aa495237518 tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:57.377284 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123f8d0a-fece-4485-b301-5aa495237518 tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:57.382627 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00390832-39db-422b-b306-cf2e321990ba tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0c64ced070444e832d9acfdea70939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19898f1737114fb7ac338122150aa72e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:57.383516 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00390832-39db-422b-b306-cf2e321990ba tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:57.383834 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00390832-39db-422b-b306-cf2e321990ba tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:57.384110 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00390832-39db-422b-b306-cf2e321990ba tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] PATH_INFO: `/v3/projects/91b9d86dd1ea40fd834ded964df9e01d` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:57.385366 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00390832-39db-422b-b306-cf2e321990ba tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] RBAC: Authorizing `identity:delete_project(project_id=91b9d86dd1ea40fd834ded964df9e01d)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:57.387346 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00390832-39db-422b-b306-cf2e321990ba tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:57.434533 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-00390832-39db-422b-b306-cf2e321990ba tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91b9d86dd1ea40fd834ded964df9e01d', 'request_id': 'req-00390832-39db-422b-b306-cf2e321990ba'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:57.442093 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-00390832-39db-422b-b306-cf2e321990ba tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91b9d86dd1ea40fd834ded964df9e01d', 'request_id': 'req-00390832-39db-422b-b306-cf2e321990ba'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:57.449450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-00390832-39db-422b-b306-cf2e321990ba tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91b9d86dd1ea40fd834ded964df9e01d', 'request_id': 'req-00390832-39db-422b-b306-cf2e321990ba'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:57.460334 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-00390832-39db-422b-b306-cf2e321990ba tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] The token cache is being invalidate because project 91b9d86dd1ea40fd834ded964df9e01d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:57.462063 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-00390832-39db-422b-b306-cf2e321990ba tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:57.462063 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-00390832-39db-422b-b306-cf2e321990ba tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:57.463082 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 964/1925] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:57 2022] DELETE /identity/v3/projects/91b9d86dd1ea40fd834ded964df9e01d => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:57.678083 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79c867-0b72-4bb0-b507-ce68c40df2af tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:57.679857 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ca79c867-0b72-4bb0-b507-ce68c40df2af tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:57.708141 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:57.748576 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79c867-0b72-4bb0-b507-ce68c40df2af tempest-VolumeQuotaClassesTest-1961834466 tempest-VolumeQuotaClassesTest-1961834466-project-admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:57.811197 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4cf2fd-2746-48c2-ae91-66d286b499b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:57.812103 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cf2fd-2746-48c2-ae91-66d286b499b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:57.812386 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cf2fd-2746-48c2-ae91-66d286b499b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:57.812660 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cf2fd-2746-48c2-ae91-66d286b499b4 admin admin] PATH_INFO: `/v3/users/4967ed0bde3f42e0a9119b54d99db1eb` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:57.813837 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4cf2fd-2746-48c2-ae91-66d286b499b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4967ed0bde3f42e0a9119b54d99db1eb)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:57.815806 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4cf2fd-2746-48c2-ae91-66d286b499b4 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:57.860857 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0d4cf2fd-2746-48c2-ae91-66d286b499b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4967ed0bde3f42e0a9119b54d99db1eb'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:57.861509 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0d4cf2fd-2746-48c2-ae91-66d286b499b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4967ed0bde3f42e0a9119b54d99db1eb'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:57.874685 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0d4cf2fd-2746-48c2-ae91-66d286b499b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4967ed0bde3f42e0a9119b54d99db1eb'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:57.882830 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0d4cf2fd-2746-48c2-ae91-66d286b499b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4967ed0bde3f42e0a9119b54d99db1eb'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:57.890023 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0d4cf2fd-2746-48c2-ae91-66d286b499b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4967ed0bde3f42e0a9119b54d99db1eb'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:57.896727 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 962/1926] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:57 2022] DELETE /identity/v3/users/4967ed0bde3f42e0a9119b54d99db1eb => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:57.901645 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632696b-847c-4d2f-acbe-8ca4e5ed1d70 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:57.902131 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632696b-847c-4d2f-acbe-8ca4e5ed1d70 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:57.902319 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632696b-847c-4d2f-acbe-8ca4e5ed1d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:57.920471 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-3632696b-847c-4d2f-acbe-8ca4e5ed1d70 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:57.921684 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-3632696b-847c-4d2f-acbe-8ca4e5ed1d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:57.985056 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 965/1927] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:57.995763 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4cf2fd-2746-48c2-ae91-66d286b499b4 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:57.997180 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-0d4cf2fd-2746-48c2-ae91-66d286b499b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:58.054152 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:58.057880 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4cf2fd-2746-48c2-ae91-66d286b499b4 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:58.062328 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f674bebc-953d-4a36-9a57-8d7a27f3296a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:58.063516 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674bebc-953d-4a36-9a57-8d7a27f3296a service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:58.063857 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674bebc-953d-4a36-9a57-8d7a27f3296a service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:58.064166 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674bebc-953d-4a36-9a57-8d7a27f3296a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:58.073819 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f674bebc-953d-4a36-9a57-8d7a27f3296a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:58.075280 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f674bebc-953d-4a36-9a57-8d7a27f3296a service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:58.085688 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 963/1928] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:58.447253 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632696b-847c-4d2f-acbe-8ca4e5ed1d70 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:58.448764 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-3632696b-847c-4d2f-acbe-8ca4e5ed1d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:58.460916 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632696b-847c-4d2f-acbe-8ca4e5ed1d70 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:58.465474 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a8502f-9f00-4e32-b898-dfcc57f18035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:58.466385 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8502f-9f00-4e32-b898-dfcc57f18035 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:58.466799 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8502f-9f00-4e32-b898-dfcc57f18035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:58.466799 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8502f-9f00-4e32-b898-dfcc57f18035 admin admin] PATH_INFO: `/v3/projects/924f76839cdf4e63bd0377990a4a0f0a` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:58.468158 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a8502f-9f00-4e32-b898-dfcc57f18035 admin admin] RBAC: Authorizing `identity:delete_project(project_id=924f76839cdf4e63bd0377990a4a0f0a)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:58.470372 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a8502f-9f00-4e32-b898-dfcc57f18035 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:58.649838 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-00a8502f-9f00-4e32-b898-dfcc57f18035 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '924f76839cdf4e63bd0377990a4a0f0a', 'request_id': 'req-00a8502f-9f00-4e32-b898-dfcc57f18035'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:58.658403 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-00a8502f-9f00-4e32-b898-dfcc57f18035 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '924f76839cdf4e63bd0377990a4a0f0a', 'request_id': 'req-00a8502f-9f00-4e32-b898-dfcc57f18035'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:58.660776 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=85,keystone:INSERT=3,keystone:DELETE=12 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:58.667220 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-00a8502f-9f00-4e32-b898-dfcc57f18035 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '924f76839cdf4e63bd0377990a4a0f0a', 'request_id': 'req-00a8502f-9f00-4e32-b898-dfcc57f18035'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:58.680531 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-00a8502f-9f00-4e32-b898-dfcc57f18035 admin admin] The token cache is being invalidate because project 924f76839cdf4e63bd0377990a4a0f0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:58.680821 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-00a8502f-9f00-4e32-b898-dfcc57f18035 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:58.681552 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-00a8502f-9f00-4e32-b898-dfcc57f18035 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:58.683074 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 966/1929] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:58 2022] DELETE /identity/v3/projects/924f76839cdf4e63bd0377990a4a0f0a => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:58.687169 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f674bebc-953d-4a36-9a57-8d7a27f3296a service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:58.700997 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f674bebc-953d-4a36-9a57-8d7a27f3296a service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:58.761611 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648365c2-73bc-4a41-8a6b-1b80d2618147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:58.762516 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648365c2-73bc-4a41-8a6b-1b80d2618147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:58.762802 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648365c2-73bc-4a41-8a6b-1b80d2618147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:58.763110 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648365c2-73bc-4a41-8a6b-1b80d2618147 admin admin] PATH_INFO: `/v3/users/6f0c64ced070444e832d9acfdea70939` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:58.764315 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648365c2-73bc-4a41-8a6b-1b80d2618147 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f0c64ced070444e832d9acfdea70939)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:58.766263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648365c2-73bc-4a41-8a6b-1b80d2618147 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:58.816516 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-648365c2-73bc-4a41-8a6b-1b80d2618147 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f0c64ced070444e832d9acfdea70939'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:58.817248 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-648365c2-73bc-4a41-8a6b-1b80d2618147 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f0c64ced070444e832d9acfdea70939'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:58.829416 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-648365c2-73bc-4a41-8a6b-1b80d2618147 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f0c64ced070444e832d9acfdea70939'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:58.838846 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-648365c2-73bc-4a41-8a6b-1b80d2618147 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f0c64ced070444e832d9acfdea70939'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:58.850549 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-648365c2-73bc-4a41-8a6b-1b80d2618147 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f0c64ced070444e832d9acfdea70939'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:58.860931 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 964/1930] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:58 2022] DELETE /identity/v3/users/6f0c64ced070444e832d9acfdea70939 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:59.160416 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a8502f-9f00-4e32-b898-dfcc57f18035 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:59.162748 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-00a8502f-9f00-4e32-b898-dfcc57f18035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:59.181177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a8502f-9f00-4e32-b898-dfcc57f18035 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:59.245794 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621f7ee-2c10-48b4-8e39-447805f8a02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:59.246654 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621f7ee-2c10-48b4-8e39-447805f8a02a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:59.246963 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621f7ee-2c10-48b4-8e39-447805f8a02a admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:59.247265 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621f7ee-2c10-48b4-8e39-447805f8a02a admin admin] PATH_INFO: `/v3/projects/19898f1737114fb7ac338122150aa72e` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:59.248454 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621f7ee-2c10-48b4-8e39-447805f8a02a admin admin] RBAC: Authorizing `identity:delete_project(project_id=19898f1737114fb7ac338122150aa72e)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:59.250330 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621f7ee-2c10-48b4-8e39-447805f8a02a admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:59.301268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-1621f7ee-2c10-48b4-8e39-447805f8a02a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19898f1737114fb7ac338122150aa72e', 'request_id': 'req-1621f7ee-2c10-48b4-8e39-447805f8a02a'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:59.309314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-1621f7ee-2c10-48b4-8e39-447805f8a02a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19898f1737114fb7ac338122150aa72e', 'request_id': 'req-1621f7ee-2c10-48b4-8e39-447805f8a02a'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:59.315833 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-1621f7ee-2c10-48b4-8e39-447805f8a02a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19898f1737114fb7ac338122150aa72e', 'request_id': 'req-1621f7ee-2c10-48b4-8e39-447805f8a02a'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:59.326456 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-1621f7ee-2c10-48b4-8e39-447805f8a02a admin admin] The token cache is being invalidate because project 19898f1737114fb7ac338122150aa72e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:59.326743 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-1621f7ee-2c10-48b4-8e39-447805f8a02a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:59.327407 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-1621f7ee-2c10-48b4-8e39-447805f8a02a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:59.329182 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 967/1931] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:59 2022] DELETE /identity/v3/projects/19898f1737114fb7ac338122150aa72e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:59.333876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfad8d5-98a3-4784-9d7d-78cf6a718b26 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:59.334174 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfad8d5-98a3-4784-9d7d-78cf6a718b26 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:59.334458 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfad8d5-98a3-4784-9d7d-78cf6a718b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:59.352835 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-5dfad8d5-98a3-4784-9d7d-78cf6a718b26 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:59.354039 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-5dfad8d5-98a3-4784-9d7d-78cf6a718b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:59.415848 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 965/1932] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:59.424003 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe54c3-d22c-46c6-a041-dcea4a73b486 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:59.424299 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe54c3-d22c-46c6-a041-dcea4a73b486 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:59.424582 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe54c3-d22c-46c6-a041-dcea4a73b486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:59.443457 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-20fe54c3-d22c-46c6-a041-dcea4a73b486 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:59.444631 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-20fe54c3-d22c-46c6-a041-dcea4a73b486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:59.452780 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 968/1933] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:59.457025 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfad8d5-98a3-4784-9d7d-78cf6a718b26 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:59.467572 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfad8d5-98a3-4784-9d7d-78cf6a718b26 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:59.472441 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42718ba2-d746-4bfc-ae04-0a283644af22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:59.473443 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42718ba2-d746-4bfc-ae04-0a283644af22 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:59.473797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42718ba2-d746-4bfc-ae04-0a283644af22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:59.474810 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42718ba2-d746-4bfc-ae04-0a283644af22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:59.475591 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42718ba2-d746-4bfc-ae04-0a283644af22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:43:59.475900 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42718ba2-d746-4bfc-ae04-0a283644af22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:59.477529 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42718ba2-d746-4bfc-ae04-0a283644af22 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:59.484695 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:59.491190 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 966/1934] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:43:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:43:59.496091 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe54c3-d22c-46c6-a041-dcea4a73b486 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:59.507199 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe54c3-d22c-46c6-a041-dcea4a73b486 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:59.512136 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf60bfe-21bf-43cb-a2de-9452d3c31786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:59.512795 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf60bfe-21bf-43cb-a2de-9452d3c31786 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:59.513078 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf60bfe-21bf-43cb-a2de-9452d3c31786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:59.513354 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf60bfe-21bf-43cb-a2de-9452d3c31786 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:59.514044 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf60bfe-21bf-43cb-a2de-9452d3c31786 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:59.516188 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf60bfe-21bf-43cb-a2de-9452d3c31786 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:59.560959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 969/1935] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:59 2022] POST /identity/v3/projects => generated 387 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:59.563205 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42718ba2-d746-4bfc-ae04-0a283644af22 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:59.579664 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42718ba2-d746-4bfc-ae04-0a283644af22 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:59.655209 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3178d9-99ca-4cd3-9845-e5e6db1438f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:59.655899 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3178d9-99ca-4cd3-9845-e5e6db1438f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:59.656188 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3178d9-99ca-4cd3-9845-e5e6db1438f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:59.656469 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3178d9-99ca-4cd3-9845-e5e6db1438f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:59.657178 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3178d9-99ca-4cd3-9845-e5e6db1438f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:59.659319 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3178d9-99ca-4cd3-9845-e5e6db1438f6 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:59.679802 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 967/1936] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:59 2022] POST /identity/v3/users => generated 294 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:43:59.683958 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf60bfe-21bf-43cb-a2de-9452d3c31786 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:59.694964 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf60bfe-21bf-43cb-a2de-9452d3c31786 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:59.699731 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be61d8-eeff-4bad-beef-dc187dc3aa0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:59.700470 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be61d8-eeff-4bad-beef-dc187dc3aa0d admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:59.700821 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be61d8-eeff-4bad-beef-dc187dc3aa0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:59.701032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be61d8-eeff-4bad-beef-dc187dc3aa0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:59.701798 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be61d8-eeff-4bad-beef-dc187dc3aa0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:59.703633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be61d8-eeff-4bad-beef-dc187dc3aa0d admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:59.715119 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 970/1937] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:59.719275 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3178d9-99ca-4cd3-9845-e5e6db1438f6 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:59.730984 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3178d9-99ca-4cd3-9845-e5e6db1438f6 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:59.735628 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ab4a8-b1a2-4909-b62e-47b7fa276459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:59.736457 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ab4a8-b1a2-4909-b62e-47b7fa276459 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:59.736744 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ab4a8-b1a2-4909-b62e-47b7fa276459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:59.737007 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ab4a8-b1a2-4909-b62e-47b7fa276459 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:59.737789 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ab4a8-b1a2-4909-b62e-47b7fa276459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:59.740147 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ab4a8-b1a2-4909-b62e-47b7fa276459 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:59.752645 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 968/1938] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:59.757493 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be61d8-eeff-4bad-beef-dc187dc3aa0d admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:59.767377 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be61d8-eeff-4bad-beef-dc187dc3aa0d admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:59.772764 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5bd56-66ef-45ac-92db-0b773661fba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:59.773645 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5bd56-66ef-45ac-92db-0b773661fba6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:59.773929 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5bd56-66ef-45ac-92db-0b773661fba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:59.774380 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5bd56-66ef-45ac-92db-0b773661fba6 admin admin] PATH_INFO: `/v3/projects/428077eedd8243d9bbb79f60762ebd4a/users/e49ef9c647d44fcaa82b72728fdf7194/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:59.790171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f5bd56-66ef-45ac-92db-0b773661fba6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=428077eedd8243d9bbb79f60762ebd4a, user_id=e49ef9c647d44fcaa82b72728fdf7194, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:59.793577 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f5bd56-66ef-45ac-92db-0b773661fba6 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:59.808966 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 971/1939] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:59 2022] PUT /identity/v3/projects/428077eedd8243d9bbb79f60762ebd4a/users/e49ef9c647d44fcaa82b72728fdf7194/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:59.820091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b756d-5c46-4b5e-90ed-9593845ca180 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:59.820432 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b756d-5c46-4b5e-90ed-9593845ca180 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:59.820746 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b756d-5c46-4b5e-90ed-9593845ca180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:59.846904 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-6c0b756d-5c46-4b5e-90ed-9593845ca180 None None] MFA Rules not processed for user `e49ef9c647d44fcaa82b72728fdf7194`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:59.848140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-6c0b756d-5c46-4b5e-90ed-9593845ca180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:59.916256 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 969/1940] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:43:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:59.920774 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5bd56-66ef-45ac-92db-0b773661fba6 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:59.934560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:59.991285 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5bd56-66ef-45ac-92db-0b773661fba6 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:59.995894 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82933bff-8ea6-4efb-b856-d55a7398ba8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:59.996544 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82933bff-8ea6-4efb-b856-d55a7398ba8c admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:59.996819 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82933bff-8ea6-4efb-b856-d55a7398ba8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:59.997095 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82933bff-8ea6-4efb-b856-d55a7398ba8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:59.997781 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82933bff-8ea6-4efb-b856-d55a7398ba8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:59.999794 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82933bff-8ea6-4efb-b856-d55a7398ba8c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:00.025220 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 972/1941] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:59 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:44:00.029581 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b756d-5c46-4b5e-90ed-9593845ca180 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:00.041154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b756d-5c46-4b5e-90ed-9593845ca180 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:00.097363 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d99fb-b38f-410e-8c97-a6a346ffa723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:00.099707 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d99fb-b38f-410e-8c97-a6a346ffa723 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:00.099991 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d99fb-b38f-410e-8c97-a6a346ffa723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:00.100270 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d99fb-b38f-410e-8c97-a6a346ffa723 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:00.101497 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99d99fb-b38f-410e-8c97-a6a346ffa723 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:00.105309 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99d99fb-b38f-410e-8c97-a6a346ffa723 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:00.116830 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:00.128094 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 970/1942] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:00 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:00.132784 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82933bff-8ea6-4efb-b856-d55a7398ba8c admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:00.142847 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82933bff-8ea6-4efb-b856-d55a7398ba8c admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:00.148230 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e520d3-7b83-483e-b439-35cdb133dd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:00.148897 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e520d3-7b83-483e-b439-35cdb133dd4f admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:00.149187 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e520d3-7b83-483e-b439-35cdb133dd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:00.149461 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e520d3-7b83-483e-b439-35cdb133dd4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:00.150193 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e520d3-7b83-483e-b439-35cdb133dd4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:00.152125 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e520d3-7b83-483e-b439-35cdb133dd4f admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:00.162896 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 973/1943] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:00.167024 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d99fb-b38f-410e-8c97-a6a346ffa723 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:00.177724 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d99fb-b38f-410e-8c97-a6a346ffa723 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:00.182458 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51484a2f-8b9e-427c-a0dd-0c2c70a989e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:00.183452 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51484a2f-8b9e-427c-a0dd-0c2c70a989e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:00.183735 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51484a2f-8b9e-427c-a0dd-0c2c70a989e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:00.184009 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51484a2f-8b9e-427c-a0dd-0c2c70a989e5 admin admin] PATH_INFO: `/v3/projects/2149887d919d4e3f9c352921f4ea2dc9/users/d8bb679ad14a4106b56b52de9403d8e7/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:00.200107 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51484a2f-8b9e-427c-a0dd-0c2c70a989e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2149887d919d4e3f9c352921f4ea2dc9, user_id=d8bb679ad14a4106b56b52de9403d8e7, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:00.202773 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51484a2f-8b9e-427c-a0dd-0c2c70a989e5 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:00.219209 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 971/1944] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:00 2022] PUT /identity/v3/projects/2149887d919d4e3f9c352921f4ea2dc9/users/d8bb679ad14a4106b56b52de9403d8e7/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:00.222968 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e520d3-7b83-483e-b439-35cdb133dd4f admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:00.233088 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e520d3-7b83-483e-b439-35cdb133dd4f admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:00.291838 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3ce0c1-d7e5-4f24-b543-691f9a6a88f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:00.292518 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ce0c1-d7e5-4f24-b543-691f9a6a88f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:00.292803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ce0c1-d7e5-4f24-b543-691f9a6a88f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:00.293079 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ce0c1-d7e5-4f24-b543-691f9a6a88f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:00.293810 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3ce0c1-d7e5-4f24-b543-691f9a6a88f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:00.295459 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3ce0c1-d7e5-4f24-b543-691f9a6a88f5 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:00.308012 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 974/1945] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:00.311384 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51484a2f-8b9e-427c-a0dd-0c2c70a989e5 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:00.322658 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51484a2f-8b9e-427c-a0dd-0c2c70a989e5 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:00.327791 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a45420a-4a11-45fd-8d6c-f7d42a9b7819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:00.328706 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45420a-4a11-45fd-8d6c-f7d42a9b7819 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:00.328972 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45420a-4a11-45fd-8d6c-f7d42a9b7819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:00.329245 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45420a-4a11-45fd-8d6c-f7d42a9b7819 admin admin] PATH_INFO: `/v3/projects/2149887d919d4e3f9c352921f4ea2dc9/users/d8bb679ad14a4106b56b52de9403d8e7/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:00.331283 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a45420a-4a11-45fd-8d6c-f7d42a9b7819 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2149887d919d4e3f9c352921f4ea2dc9, user_id=d8bb679ad14a4106b56b52de9403d8e7, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:00.335248 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a45420a-4a11-45fd-8d6c-f7d42a9b7819 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:00.346211 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 972/1946] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:00 2022] PUT /identity/v3/projects/2149887d919d4e3f9c352921f4ea2dc9/users/d8bb679ad14a4106b56b52de9403d8e7/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:00.358507 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ba0c9-d579-42be-b957-599243a3ec41 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:00.358837 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ba0c9-d579-42be-b957-599243a3ec41 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:00.359235 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ba0c9-d579-42be-b957-599243a3ec41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:00.388463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-6a0ba0c9-d579-42be-b957-599243a3ec41 None None] MFA Rules not processed for user `d8bb679ad14a4106b56b52de9403d8e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:00.390122 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6a0ba0c9-d579-42be-b957-599243a3ec41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:00.483391 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 975/1947] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:00 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:00.497639 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a45420a-4a11-45fd-8d6c-f7d42a9b7819 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:00.500426 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-6a45420a-4a11-45fd-8d6c-f7d42a9b7819 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:00.578072 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a45420a-4a11-45fd-8d6c-f7d42a9b7819 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:00.582919 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05431be3-2c0f-4842-b47f-9da437a481b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:00.582919 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05431be3-2c0f-4842-b47f-9da437a481b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:00.582919 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05431be3-2c0f-4842-b47f-9da437a481b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:00.582919 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05431be3-2c0f-4842-b47f-9da437a481b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:00.592313 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05431be3-2c0f-4842-b47f-9da437a481b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:00.593768 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05431be3-2c0f-4842-b47f-9da437a481b3 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:00.603550 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 973/1948] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:00 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:01.499912 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ba0c9-d579-42be-b957-599243a3ec41 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:01.511659 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:01.590044 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ba0c9-d579-42be-b957-599243a3ec41 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:01.591206 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05431be3-2c0f-4842-b47f-9da437a481b3 service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:01.592701 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-05431be3-2c0f-4842-b47f-9da437a481b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:01.597842 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6c458c-1fdf-4345-8dc5-84838c591491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:01.599438 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c458c-1fdf-4345-8dc5-84838c591491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:01.599649 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c458c-1fdf-4345-8dc5-84838c591491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:01.599874 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c458c-1fdf-4345-8dc5-84838c591491 admin admin] PATH_INFO: `/v3/users/e49ef9c647d44fcaa82b72728fdf7194` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:01.601100 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6c458c-1fdf-4345-8dc5-84838c591491 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e49ef9c647d44fcaa82b72728fdf7194)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:01.603158 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6c458c-1fdf-4345-8dc5-84838c591491 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:01.608412 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05431be3-2c0f-4842-b47f-9da437a481b3 service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:01.615697 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4d1ac7-71a2-48a6-81f8-1d57e3be8570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:01.617072 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d1ac7-71a2-48a6-81f8-1d57e3be8570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:01.617555 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d1ac7-71a2-48a6-81f8-1d57e3be8570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:01.618075 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d1ac7-71a2-48a6-81f8-1d57e3be8570 admin admin] PATH_INFO: `/v3/users/db037e07d96746b78c3e67376649a657` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:01.619572 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4d1ac7-71a2-48a6-81f8-1d57e3be8570 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db037e07d96746b78c3e67376649a657)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:01.621974 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4d1ac7-71a2-48a6-81f8-1d57e3be8570 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:01.672747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-af6c458c-1fdf-4345-8dc5-84838c591491 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e49ef9c647d44fcaa82b72728fdf7194'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:01.673781 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-af6c458c-1fdf-4345-8dc5-84838c591491 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e49ef9c647d44fcaa82b72728fdf7194'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:01.678838 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-4d4d1ac7-71a2-48a6-81f8-1d57e3be8570 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db037e07d96746b78c3e67376649a657'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:01.683431 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-4d4d1ac7-71a2-48a6-81f8-1d57e3be8570 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db037e07d96746b78c3e67376649a657'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:01.690862 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-af6c458c-1fdf-4345-8dc5-84838c591491 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e49ef9c647d44fcaa82b72728fdf7194'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:01.691694 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-af6c458c-1fdf-4345-8dc5-84838c591491 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e49ef9c647d44fcaa82b72728fdf7194'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:01.700561 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-4d4d1ac7-71a2-48a6-81f8-1d57e3be8570 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db037e07d96746b78c3e67376649a657'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:01.702396 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-af6c458c-1fdf-4345-8dc5-84838c591491 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e49ef9c647d44fcaa82b72728fdf7194'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:01.712376 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-4d4d1ac7-71a2-48a6-81f8-1d57e3be8570 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db037e07d96746b78c3e67376649a657'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:01.727088 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 976/1949] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:01 2022] DELETE /identity/v3/users/e49ef9c647d44fcaa82b72728fdf7194 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:01.727271 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-4d4d1ac7-71a2-48a6-81f8-1d57e3be8570 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db037e07d96746b78c3e67376649a657'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:01.728497 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c5ecb-c7b2-44c3-adb2-f01cc4054d01 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:01.728790 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c5ecb-c7b2-44c3-adb2-f01cc4054d01 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:01.729069 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c5ecb-c7b2-44c3-adb2-f01cc4054d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:01.735186 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 974/1950] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:01 2022] DELETE /identity/v3/users/db037e07d96746b78c3e67376649a657 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:01.739818 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b30f5-7014-4c19-850c-7affaa9125a1 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:01.740113 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b30f5-7014-4c19-850c-7affaa9125a1 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:01.740399 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b30f5-7014-4c19-850c-7affaa9125a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:01.755864 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-6f5c5ecb-c7b2-44c3-adb2-f01cc4054d01 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:01.757084 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6f5c5ecb-c7b2-44c3-adb2-f01cc4054d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:01.758894 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-a13b30f5-7014-4c19-850c-7affaa9125a1 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:01.760217 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-a13b30f5-7014-4c19-850c-7affaa9125a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:01.793350 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:01.823736 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 977/1951] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:44:01.828313 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 975/1952] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:44:01.834973 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c5ecb-c7b2-44c3-adb2-f01cc4054d01 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:01.836376 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6f5c5ecb-c7b2-44c3-adb2-f01cc4054d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:01.837926 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b30f5-7014-4c19-850c-7affaa9125a1 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:01.839349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-a13b30f5-7014-4c19-850c-7affaa9125a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:01.871824 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:01.904245 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b30f5-7014-4c19-850c-7affaa9125a1 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:01.908007 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c5ecb-c7b2-44c3-adb2-f01cc4054d01 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:01.908124 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e302a-9561-4266-ba9b-840a5c358484 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:01.908759 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e302a-9561-4266-ba9b-840a5c358484 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:01.908950 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e302a-9561-4266-ba9b-840a5c358484 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:01.909239 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e302a-9561-4266-ba9b-840a5c358484 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:01.912797 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576036ff-4f85-45a9-b0ad-2dc13eae880b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:01.913462 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576036ff-4f85-45a9-b0ad-2dc13eae880b service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:01.913747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576036ff-4f85-45a9-b0ad-2dc13eae880b service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:01.914026 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576036ff-4f85-45a9-b0ad-2dc13eae880b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:01.920401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e302a-9561-4266-ba9b-840a5c358484 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:01.922201 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e302a-9561-4266-ba9b-840a5c358484 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:01.925764 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576036ff-4f85-45a9-b0ad-2dc13eae880b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:01.930232 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576036ff-4f85-45a9-b0ad-2dc13eae880b service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:01.934149 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 976/1953] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:44:01.934984 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e302a-9561-4266-ba9b-840a5c358484 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:01.936429 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-8e4e302a-9561-4266-ba9b-840a5c358484 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:01.939881 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 978/1954] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:01.953564 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e302a-9561-4266-ba9b-840a5c358484 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:01.959127 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d47987-7ad5-4c8c-b400-92564445788d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:01.960024 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d47987-7ad5-4c8c-b400-92564445788d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:01.960533 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d47987-7ad5-4c8c-b400-92564445788d admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:01.960644 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d47987-7ad5-4c8c-b400-92564445788d admin admin] PATH_INFO: `/v3/users/82c24de3fe5d48029801df2127b24590` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:01.963699 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d47987-7ad5-4c8c-b400-92564445788d admin admin] RBAC: Authorizing `identity:delete_user(user_id=82c24de3fe5d48029801df2127b24590)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:01.964598 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d47987-7ad5-4c8c-b400-92564445788d admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:02.027103 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-14d47987-7ad5-4c8c-b400-92564445788d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82c24de3fe5d48029801df2127b24590'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:02.027791 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-14d47987-7ad5-4c8c-b400-92564445788d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82c24de3fe5d48029801df2127b24590'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:02.041640 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-14d47987-7ad5-4c8c-b400-92564445788d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82c24de3fe5d48029801df2127b24590'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:02.050355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-14d47987-7ad5-4c8c-b400-92564445788d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82c24de3fe5d48029801df2127b24590'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:02.061078 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-14d47987-7ad5-4c8c-b400-92564445788d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82c24de3fe5d48029801df2127b24590'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:02.068716 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 977/1955] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:01 2022] DELETE /identity/v3/users/82c24de3fe5d48029801df2127b24590 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:02.075563 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf60517-b179-4109-a1f3-d7a17cef08ca None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:02.076144 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf60517-b179-4109-a1f3-d7a17cef08ca None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:02.076721 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf60517-b179-4109-a1f3-d7a17cef08ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:02.105658 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-acf60517-b179-4109-a1f3-d7a17cef08ca None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:02.107947 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-acf60517-b179-4109-a1f3-d7a17cef08ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:02.186917 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 979/1956] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:02.199048 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d47987-7ad5-4c8c-b400-92564445788d admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:02.201178 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-14d47987-7ad5-4c8c-b400-92564445788d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:02.265916 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf60517-b179-4109-a1f3-d7a17cef08ca None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:02.267865 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-acf60517-b179-4109-a1f3-d7a17cef08ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:02.282971 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf60517-b179-4109-a1f3-d7a17cef08ca None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:02.290837 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a0c318-d4d8-4e45-8800-3dc9a864c553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:02.294951 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0c318-d4d8-4e45-8800-3dc9a864c553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:02.295270 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0c318-d4d8-4e45-8800-3dc9a864c553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:02.295554 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0c318-d4d8-4e45-8800-3dc9a864c553 admin admin] PATH_INFO: `/v3/projects/428077eedd8243d9bbb79f60762ebd4a` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:02.298631 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:02.299162 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a0c318-d4d8-4e45-8800-3dc9a864c553 admin admin] RBAC: Authorizing `identity:delete_project(project_id=428077eedd8243d9bbb79f60762ebd4a)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:02.303421 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a0c318-d4d8-4e45-8800-3dc9a864c553 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:02.309025 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d47987-7ad5-4c8c-b400-92564445788d admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:02.313165 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c040ca-0edf-4214-876b-ffe5b96c7355 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:02.313739 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c040ca-0edf-4214-876b-ffe5b96c7355 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:02.314332 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c040ca-0edf-4214-876b-ffe5b96c7355 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:02.314433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c040ca-0edf-4214-876b-ffe5b96c7355 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:02.326643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c040ca-0edf-4214-876b-ffe5b96c7355 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:02.330078 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c040ca-0edf-4214-876b-ffe5b96c7355 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:02.340856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 978/1957] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:02.353659 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c040ca-0edf-4214-876b-ffe5b96c7355 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:02.358986 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-36c040ca-0edf-4214-876b-ffe5b96c7355 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:02.374187 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b8a0c318-d4d8-4e45-8800-3dc9a864c553 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '428077eedd8243d9bbb79f60762ebd4a', 'request_id': 'req-b8a0c318-d4d8-4e45-8800-3dc9a864c553'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:02.385985 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c040ca-0edf-4214-876b-ffe5b96c7355 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:02.387357 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b8a0c318-d4d8-4e45-8800-3dc9a864c553 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '428077eedd8243d9bbb79f60762ebd4a', 'request_id': 'req-b8a0c318-d4d8-4e45-8800-3dc9a864c553'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:02.398558 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b8a0c318-d4d8-4e45-8800-3dc9a864c553 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '428077eedd8243d9bbb79f60762ebd4a', 'request_id': 'req-b8a0c318-d4d8-4e45-8800-3dc9a864c553'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:02.416412 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b8a0c318-d4d8-4e45-8800-3dc9a864c553 admin admin] The token cache is being invalidate because project 428077eedd8243d9bbb79f60762ebd4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:02.416412 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b8a0c318-d4d8-4e45-8800-3dc9a864c553 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:02.417212 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b8a0c318-d4d8-4e45-8800-3dc9a864c553 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:02.418828 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 980/1958] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:02 2022] DELETE /identity/v3/projects/428077eedd8243d9bbb79f60762ebd4a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:02.423185 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a0c318-d4d8-4e45-8800-3dc9a864c553 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:02.424978 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-b8a0c318-d4d8-4e45-8800-3dc9a864c553 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:02.464761 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c25c5c-100f-43c8-ae4a-94a72b1132ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:02.465628 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c25c5c-100f-43c8-ae4a-94a72b1132ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:02.466371 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c25c5c-100f-43c8-ae4a-94a72b1132ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:02.466659 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c25c5c-100f-43c8-ae4a-94a72b1132ba admin admin] PATH_INFO: `/v3/projects/71b718b10be64311a6fc49c19b937c58` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:02.467896 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c25c5c-100f-43c8-ae4a-94a72b1132ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=71b718b10be64311a6fc49c19b937c58)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:02.469794 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c25c5c-100f-43c8-ae4a-94a72b1132ba admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:02.496467 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:02.515734 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a0c318-d4d8-4e45-8800-3dc9a864c553 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:02.536815 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-d7c25c5c-100f-43c8-ae4a-94a72b1132ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71b718b10be64311a6fc49c19b937c58', 'request_id': 'req-d7c25c5c-100f-43c8-ae4a-94a72b1132ba'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:02.546158 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-d7c25c5c-100f-43c8-ae4a-94a72b1132ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71b718b10be64311a6fc49c19b937c58', 'request_id': 'req-d7c25c5c-100f-43c8-ae4a-94a72b1132ba'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:02.553503 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-d7c25c5c-100f-43c8-ae4a-94a72b1132ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71b718b10be64311a6fc49c19b937c58', 'request_id': 'req-d7c25c5c-100f-43c8-ae4a-94a72b1132ba'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:02.570297 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-d7c25c5c-100f-43c8-ae4a-94a72b1132ba admin admin] The token cache is being invalidate because project 71b718b10be64311a6fc49c19b937c58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:02.570297 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-d7c25c5c-100f-43c8-ae4a-94a72b1132ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:02.570955 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-d7c25c5c-100f-43c8-ae4a-94a72b1132ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:02.573034 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 979/1959] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:02 2022] DELETE /identity/v3/projects/71b718b10be64311a6fc49c19b937c58 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:02.587196 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c25c5c-100f-43c8-ae4a-94a72b1132ba admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:02.590234 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0188f4-9691-4e63-a686-ed5aeca7c8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:02.590507 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d7c25c5c-100f-43c8-ae4a-94a72b1132ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:02.591540 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0188f4-9691-4e63-a686-ed5aeca7c8f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:02.591540 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0188f4-9691-4e63-a686-ed5aeca7c8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:02.591921 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0188f4-9691-4e63-a686-ed5aeca7c8f7 admin admin] PATH_INFO: `/v3/users/d8bb679ad14a4106b56b52de9403d8e7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:02.593483 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0188f4-9691-4e63-a686-ed5aeca7c8f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8bb679ad14a4106b56b52de9403d8e7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:02.595804 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0188f4-9691-4e63-a686-ed5aeca7c8f7 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:02.610241 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c25c5c-100f-43c8-ae4a-94a72b1132ba admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:02.616140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49474da3-1687-435b-902b-7abcc015afd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:02.617770 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49474da3-1687-435b-902b-7abcc015afd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:02.618153 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49474da3-1687-435b-902b-7abcc015afd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:02.618528 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49474da3-1687-435b-902b-7abcc015afd9 admin admin] PATH_INFO: `/v3/users/021cf3b95f8d4c358b8b61db26d65126` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:02.620180 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49474da3-1687-435b-902b-7abcc015afd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=021cf3b95f8d4c358b8b61db26d65126)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:02.624653 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49474da3-1687-435b-902b-7abcc015afd9 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:02.667069 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-5d0188f4-9691-4e63-a686-ed5aeca7c8f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8bb679ad14a4106b56b52de9403d8e7'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:02.667987 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-5d0188f4-9691-4e63-a686-ed5aeca7c8f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8bb679ad14a4106b56b52de9403d8e7'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:02.694603 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-49474da3-1687-435b-902b-7abcc015afd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '021cf3b95f8d4c358b8b61db26d65126'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:02.695342 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-49474da3-1687-435b-902b-7abcc015afd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '021cf3b95f8d4c358b8b61db26d65126'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:02.704284 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-5d0188f4-9691-4e63-a686-ed5aeca7c8f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8bb679ad14a4106b56b52de9403d8e7'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:02.705031 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-5d0188f4-9691-4e63-a686-ed5aeca7c8f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8bb679ad14a4106b56b52de9403d8e7'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:02.712527 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-49474da3-1687-435b-902b-7abcc015afd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '021cf3b95f8d4c358b8b61db26d65126'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:02.717155 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-5d0188f4-9691-4e63-a686-ed5aeca7c8f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8bb679ad14a4106b56b52de9403d8e7'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:02.722342 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-49474da3-1687-435b-902b-7abcc015afd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '021cf3b95f8d4c358b8b61db26d65126'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:02.735883 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:02.740607 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-49474da3-1687-435b-902b-7abcc015afd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '021cf3b95f8d4c358b8b61db26d65126'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:02.747873 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 981/1960] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:02 2022] DELETE /identity/v3/users/d8bb679ad14a4106b56b52de9403d8e7 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:02.750019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0188f4-9691-4e63-a686-ed5aeca7c8f7 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:02.755418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 980/1961] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:02 2022] DELETE /identity/v3/users/021cf3b95f8d4c358b8b61db26d65126 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:02.756806 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5d0188f4-9691-4e63-a686-ed5aeca7c8f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:02.838586 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0188f4-9691-4e63-a686-ed5aeca7c8f7 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:02.870544 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:02.913634 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6f179-4561-4391-afd6-d86006609121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:02.914525 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6f179-4561-4391-afd6-d86006609121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:02.915097 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6f179-4561-4391-afd6-d86006609121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:02.915097 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6f179-4561-4391-afd6-d86006609121 admin admin] PATH_INFO: `/v3/projects/f34a66af437543cf8f886bce61210759` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:02.916396 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6f179-4561-4391-afd6-d86006609121 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f34a66af437543cf8f886bce61210759)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:02.918586 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6f179-4561-4391-afd6-d86006609121 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:02.991043 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4ec6f179-4561-4391-afd6-d86006609121 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f34a66af437543cf8f886bce61210759', 'request_id': 'req-4ec6f179-4561-4391-afd6-d86006609121'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.004955 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4ec6f179-4561-4391-afd6-d86006609121 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f34a66af437543cf8f886bce61210759', 'request_id': 'req-4ec6f179-4561-4391-afd6-d86006609121'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.013663 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4ec6f179-4561-4391-afd6-d86006609121 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f34a66af437543cf8f886bce61210759', 'request_id': 'req-4ec6f179-4561-4391-afd6-d86006609121'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.038251 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4ec6f179-4561-4391-afd6-d86006609121 admin admin] The token cache is being invalidate because project f34a66af437543cf8f886bce61210759 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:03.038550 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4ec6f179-4561-4391-afd6-d86006609121 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.039337 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4ec6f179-4561-4391-afd6-d86006609121 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.042431 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 982/1962] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:02 2022] DELETE /identity/v3/projects/f34a66af437543cf8f886bce61210759 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:03.045811 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49474da3-1687-435b-902b-7abcc015afd9 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:03.047669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-49474da3-1687-435b-902b-7abcc015afd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:03.065258 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49474da3-1687-435b-902b-7abcc015afd9 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:03.181608 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83bdcb-3da8-414a-afff-49e8bca714de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:03.182512 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83bdcb-3da8-414a-afff-49e8bca714de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:03.182801 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83bdcb-3da8-414a-afff-49e8bca714de admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:03.183196 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83bdcb-3da8-414a-afff-49e8bca714de admin admin] PATH_INFO: `/v3/users/910d3360da1c414496c0eb30ce0e1293` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:03.185661 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b83bdcb-3da8-414a-afff-49e8bca714de admin admin] RBAC: Authorizing `identity:delete_user(user_id=910d3360da1c414496c0eb30ce0e1293)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:03.189183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b83bdcb-3da8-414a-afff-49e8bca714de admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:03.260088 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-9b83bdcb-3da8-414a-afff-49e8bca714de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '910d3360da1c414496c0eb30ce0e1293'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.263157 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-9b83bdcb-3da8-414a-afff-49e8bca714de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '910d3360da1c414496c0eb30ce0e1293'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.287896 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6f179-4561-4391-afd6-d86006609121 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:03.289867 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-9b83bdcb-3da8-414a-afff-49e8bca714de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '910d3360da1c414496c0eb30ce0e1293'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.290435 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-4ec6f179-4561-4391-afd6-d86006609121 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:03.321670 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6f179-4561-4391-afd6-d86006609121 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:03.323823 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-9b83bdcb-3da8-414a-afff-49e8bca714de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '910d3360da1c414496c0eb30ce0e1293'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.340223 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-9b83bdcb-3da8-414a-afff-49e8bca714de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '910d3360da1c414496c0eb30ce0e1293'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.348301 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 981/1963] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:03 2022] DELETE /identity/v3/users/910d3360da1c414496c0eb30ce0e1293 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:03.349751 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83bdcb-3da8-414a-afff-49e8bca714de admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:03.352611 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9362ae-8542-4660-b527-296d1fc04183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:03.353534 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9362ae-8542-4660-b527-296d1fc04183 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:03.353824 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9362ae-8542-4660-b527-296d1fc04183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:03.354363 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9362ae-8542-4660-b527-296d1fc04183 admin admin] PATH_INFO: `/v3/projects/2149887d919d4e3f9c352921f4ea2dc9` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:03.355691 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9362ae-8542-4660-b527-296d1fc04183 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2149887d919d4e3f9c352921f4ea2dc9)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:03.357784 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9362ae-8542-4660-b527-296d1fc04183 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:03.359131 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-9b83bdcb-3da8-414a-afff-49e8bca714de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:03.427782 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-7f9362ae-8542-4660-b527-296d1fc04183 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2149887d919d4e3f9c352921f4ea2dc9', 'request_id': 'req-7f9362ae-8542-4660-b527-296d1fc04183'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.439592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-7f9362ae-8542-4660-b527-296d1fc04183 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2149887d919d4e3f9c352921f4ea2dc9', 'request_id': 'req-7f9362ae-8542-4660-b527-296d1fc04183'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.447754 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83bdcb-3da8-414a-afff-49e8bca714de admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:03.462214 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-7f9362ae-8542-4660-b527-296d1fc04183 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2149887d919d4e3f9c352921f4ea2dc9', 'request_id': 'req-7f9362ae-8542-4660-b527-296d1fc04183'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.474548 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:03.478128 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-7f9362ae-8542-4660-b527-296d1fc04183 admin admin] The token cache is being invalidate because project 2149887d919d4e3f9c352921f4ea2dc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:03.478772 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-7f9362ae-8542-4660-b527-296d1fc04183 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.479495 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-7f9362ae-8542-4660-b527-296d1fc04183 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.481379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 983/1964] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:03 2022] DELETE /identity/v3/projects/2149887d919d4e3f9c352921f4ea2dc9 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:03.486209 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37750873-62a7-4acb-83d4-d10abd6c6af4 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:03.487015 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37750873-62a7-4acb-83d4-d10abd6c6af4 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:03.487472 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37750873-62a7-4acb-83d4-d10abd6c6af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:03.511018 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-37750873-62a7-4acb-83d4-d10abd6c6af4 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:03.512337 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-37750873-62a7-4acb-83d4-d10abd6c6af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:03.528535 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407df449-9e11-45b0-8cfb-d66d1ede85c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:03.530155 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407df449-9e11-45b0-8cfb-d66d1ede85c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:03.530530 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407df449-9e11-45b0-8cfb-d66d1ede85c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:03.530894 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407df449-9e11-45b0-8cfb-d66d1ede85c1 admin admin] PATH_INFO: `/v3/projects/e2da2cd01ea144699aea0ac22941a9ad` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:03.532431 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407df449-9e11-45b0-8cfb-d66d1ede85c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2da2cd01ea144699aea0ac22941a9ad)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:03.534487 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407df449-9e11-45b0-8cfb-d66d1ede85c1 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:03.577620 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:03.608531 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-407df449-9e11-45b0-8cfb-d66d1ede85c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2da2cd01ea144699aea0ac22941a9ad', 'request_id': 'req-407df449-9e11-45b0-8cfb-d66d1ede85c1'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.613192 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 984/1965] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:03.621428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2185535-71e1-430a-9cc3-064492e903f7 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:03.622122 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2185535-71e1-430a-9cc3-064492e903f7 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:03.622391 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2185535-71e1-430a-9cc3-064492e903f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:03.624308 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-407df449-9e11-45b0-8cfb-d66d1ede85c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2da2cd01ea144699aea0ac22941a9ad', 'request_id': 'req-407df449-9e11-45b0-8cfb-d66d1ede85c1'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.636622 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-407df449-9e11-45b0-8cfb-d66d1ede85c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2da2cd01ea144699aea0ac22941a9ad', 'request_id': 'req-407df449-9e11-45b0-8cfb-d66d1ede85c1'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.642734 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-c2185535-71e1-430a-9cc3-064492e903f7 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:03.643801 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c2185535-71e1-430a-9cc3-064492e903f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:03.651965 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-407df449-9e11-45b0-8cfb-d66d1ede85c1 admin admin] The token cache is being invalidate because project e2da2cd01ea144699aea0ac22941a9ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:03.652371 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-407df449-9e11-45b0-8cfb-d66d1ede85c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.653129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-407df449-9e11-45b0-8cfb-d66d1ede85c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.654890 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 982/1966] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:03 2022] DELETE /identity/v3/projects/e2da2cd01ea144699aea0ac22941a9ad => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:03.659796 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ddd22-bc29-497a-851e-afa01ee4f8b9 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:03.660140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ddd22-bc29-497a-851e-afa01ee4f8b9 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:03.661088 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ddd22-bc29-497a-851e-afa01ee4f8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:03.681876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-d49ddd22-bc29-497a-851e-afa01ee4f8b9 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:03.683137 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d49ddd22-bc29-497a-851e-afa01ee4f8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:03.732189 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 985/1967] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:03.733494 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2185535-71e1-430a-9cc3-064492e903f7 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:03.735425 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c2185535-71e1-430a-9cc3-064492e903f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:03.750818 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2185535-71e1-430a-9cc3-064492e903f7 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:03.755620 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4737c1c0-5db7-42f5-b853-b13866aaf2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:03.756497 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737c1c0-5db7-42f5-b853-b13866aaf2cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:03.756780 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737c1c0-5db7-42f5-b853-b13866aaf2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:03.757055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737c1c0-5db7-42f5-b853-b13866aaf2cd admin admin] PATH_INFO: `/v3/projects/af7556007c6f4035bbe9f8be21099555` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:03.758258 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4737c1c0-5db7-42f5-b853-b13866aaf2cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=af7556007c6f4035bbe9f8be21099555)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:03.760337 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4737c1c0-5db7-42f5-b853-b13866aaf2cd admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:03.780557 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 983/1968] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:03.781828 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ddd22-bc29-497a-851e-afa01ee4f8b9 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:03.798814 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ddd22-bc29-497a-851e-afa01ee4f8b9 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:03.812625 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4737c1c0-5db7-42f5-b853-b13866aaf2cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af7556007c6f4035bbe9f8be21099555', 'request_id': 'req-4737c1c0-5db7-42f5-b853-b13866aaf2cd'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.826251 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4737c1c0-5db7-42f5-b853-b13866aaf2cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af7556007c6f4035bbe9f8be21099555', 'request_id': 'req-4737c1c0-5db7-42f5-b853-b13866aaf2cd'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.838144 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4737c1c0-5db7-42f5-b853-b13866aaf2cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af7556007c6f4035bbe9f8be21099555', 'request_id': 'req-4737c1c0-5db7-42f5-b853-b13866aaf2cd'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.850969 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4737c1c0-5db7-42f5-b853-b13866aaf2cd admin admin] The token cache is being invalidate because project af7556007c6f4035bbe9f8be21099555 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:03.851293 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4737c1c0-5db7-42f5-b853-b13866aaf2cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.852044 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4737c1c0-5db7-42f5-b853-b13866aaf2cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.853922 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 986/1969] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:03 2022] DELETE /identity/v3/projects/af7556007c6f4035bbe9f8be21099555 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:03.859146 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fde527-c2c5-4d4e-a803-a2134c1210db None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:03.859481 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fde527-c2c5-4d4e-a803-a2134c1210db None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:03.860376 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fde527-c2c5-4d4e-a803-a2134c1210db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:03.872204 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:03.883573 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588d901-c9f0-4bd6-b739-7e09b6b01026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:03.884518 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588d901-c9f0-4bd6-b739-7e09b6b01026 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:03.885207 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588d901-c9f0-4bd6-b739-7e09b6b01026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:03.885499 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588d901-c9f0-4bd6-b739-7e09b6b01026 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:03.885833 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-14fde527-c2c5-4d4e-a803-a2134c1210db None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:03.886219 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588d901-c9f0-4bd6-b739-7e09b6b01026 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:44:03.886858 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588d901-c9f0-4bd6-b739-7e09b6b01026 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:03.887586 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-14fde527-c2c5-4d4e-a803-a2134c1210db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:03.889474 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588d901-c9f0-4bd6-b739-7e09b6b01026 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:03.900087 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 987/1970] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:03.901748 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c597a-90e7-4982-a476-d42ff250992b None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:03.902044 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c597a-90e7-4982-a476-d42ff250992b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:03.902611 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c597a-90e7-4982-a476-d42ff250992b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:03.917247 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 984/1971] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:44:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:44:03.919794 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588d901-c9f0-4bd6-b739-7e09b6b01026 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:03.931429 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588d901-c9f0-4bd6-b739-7e09b6b01026 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:03.937065 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-280c597a-90e7-4982-a476-d42ff250992b None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:03.937472 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb304c5a-5a4b-403f-b6d3-f9128d6acf7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:03.938301 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb304c5a-5a4b-403f-b6d3-f9128d6acf7d admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:03.938596 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb304c5a-5a4b-403f-b6d3-f9128d6acf7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:03.938703 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-280c597a-90e7-4982-a476-d42ff250992b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:03.938940 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb304c5a-5a4b-403f-b6d3-f9128d6acf7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:03.939732 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb304c5a-5a4b-403f-b6d3-f9128d6acf7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:44:03.940138 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb304c5a-5a4b-403f-b6d3-f9128d6acf7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:03.941842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb304c5a-5a4b-403f-b6d3-f9128d6acf7d admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:03.956773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 988/1972] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:03.956773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c597a-90e7-4982-a476-d42ff250992b None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:03.956773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-280c597a-90e7-4982-a476-d42ff250992b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:03.970933 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 985/1973] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:44:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:44:03.970933 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f44d1-b109-4ee3-84f1-46312f488c26 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:03.971509 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c597a-90e7-4982-a476-d42ff250992b None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:03.971653 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f44d1-b109-4ee3-84f1-46312f488c26 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:03.971653 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f44d1-b109-4ee3-84f1-46312f488c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:03.974280 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0170c304-a41f-47a2-bba5-8bcbded85aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:03.975225 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170c304-a41f-47a2-bba5-8bcbded85aa1 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:03.975390 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170c304-a41f-47a2-bba5-8bcbded85aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:03.975616 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170c304-a41f-47a2-bba5-8bcbded85aa1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:03.976317 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0170c304-a41f-47a2-bba5-8bcbded85aa1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:03.978267 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0170c304-a41f-47a2-bba5-8bcbded85aa1 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:03.993435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-d45f44d1-b109-4ee3-84f1-46312f488c26 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:03.995721 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d45f44d1-b109-4ee3-84f1-46312f488c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:04.010294 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 986/1974] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:04.011215 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f44d1-b109-4ee3-84f1-46312f488c26 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:04.015122 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 989/1975] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:03 2022] POST /identity/v3/projects => generated 381 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:04.018316 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0170c304-a41f-47a2-bba5-8bcbded85aa1 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:04.024984 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f44d1-b109-4ee3-84f1-46312f488c26 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:04.092425 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:04.106661 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aaa08e-15a0-4246-925c-893336ef513e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:04.107663 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aaa08e-15a0-4246-925c-893336ef513e admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:04.108155 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aaa08e-15a0-4246-925c-893336ef513e admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:04.108700 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aaa08e-15a0-4246-925c-893336ef513e admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:04.109606 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aaa08e-15a0-4246-925c-893336ef513e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:04.111952 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aaa08e-15a0-4246-925c-893336ef513e admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:04.146642 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0170c304-a41f-47a2-bba5-8bcbded85aa1 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:04.147115 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 987/1976] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:04 2022] POST /identity/v3/projects => generated 371 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:04.150889 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aaa08e-15a0-4246-925c-893336ef513e admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:04.232393 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b5c6b-5c99-497f-8bbd-2e5838d9247a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:04.233148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b5c6b-5c99-497f-8bbd-2e5838d9247a admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:04.233442 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b5c6b-5c99-497f-8bbd-2e5838d9247a admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:04.233729 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b5c6b-5c99-497f-8bbd-2e5838d9247a admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:04.234484 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4b5c6b-5c99-497f-8bbd-2e5838d9247a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:44:04.234799 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4b5c6b-5c99-497f-8bbd-2e5838d9247a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:04.236599 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4b5c6b-5c99-497f-8bbd-2e5838d9247a admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:04.237648 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aaa08e-15a0-4246-925c-893336ef513e admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:04.245109 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596add1b-eb48-42b2-9e52-9404865c7ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:04.245795 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596add1b-eb48-42b2-9e52-9404865c7ed9 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:04.246083 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596add1b-eb48-42b2-9e52-9404865c7ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:04.246890 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596add1b-eb48-42b2-9e52-9404865c7ed9 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:04.247635 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596add1b-eb48-42b2-9e52-9404865c7ed9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:04.250248 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596add1b-eb48-42b2-9e52-9404865c7ed9 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:04.259947 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 990/1977] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:44:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:44:04.261074 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b5c6b-5c99-497f-8bbd-2e5838d9247a admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:04.271520 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b5c6b-5c99-497f-8bbd-2e5838d9247a admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:04.276284 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d2b6e-545f-444c-9298-5ab327036955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:04.280295 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 988/1978] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:04 2022] POST /identity/v3/users => generated 291 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:04.280473 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d2b6e-545f-444c-9298-5ab327036955 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:04.280473 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d2b6e-545f-444c-9298-5ab327036955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:04.280638 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596add1b-eb48-42b2-9e52-9404865c7ed9 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:04.280730 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d2b6e-545f-444c-9298-5ab327036955 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:04.280730 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8d2b6e-545f-444c-9298-5ab327036955 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:04.280890 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8d2b6e-545f-444c-9298-5ab327036955 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:04.291663 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596add1b-eb48-42b2-9e52-9404865c7ed9 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:04.297375 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a804bf-3452-460b-b176-aee918aff740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:04.298418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a804bf-3452-460b-b176-aee918aff740 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:04.298835 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a804bf-3452-460b-b176-aee918aff740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:04.299167 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a804bf-3452-460b-b176-aee918aff740 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:04.299870 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a804bf-3452-460b-b176-aee918aff740 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:04.302369 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a804bf-3452-460b-b176-aee918aff740 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:04.303453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 991/1979] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:04 2022] POST /identity/v3/users => generated 286 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:04.304365 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d2b6e-545f-444c-9298-5ab327036955 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:04.314731 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d2b6e-545f-444c-9298-5ab327036955 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:04.320614 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446065f-a442-4976-8f6b-995e0f2f40f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:04.321394 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446065f-a442-4976-8f6b-995e0f2f40f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:04.321728 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446065f-a442-4976-8f6b-995e0f2f40f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:04.322185 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446065f-a442-4976-8f6b-995e0f2f40f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:04.322774 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3446065f-a442-4976-8f6b-995e0f2f40f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:04.324427 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3446065f-a442-4976-8f6b-995e0f2f40f4 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:04.335024 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 989/1980] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:04 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:04.335833 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a804bf-3452-460b-b176-aee918aff740 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:04.339897 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 992/1981] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:04.341848 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446065f-a442-4976-8f6b-995e0f2f40f4 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:04.353442 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446065f-a442-4976-8f6b-995e0f2f40f4 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:04.358931 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:04.420368 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a804bf-3452-460b-b176-aee918aff740 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:04.425441 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fffce3-4a6a-45a2-b1a1-574c4d25e9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:04.426076 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fffce3-4a6a-45a2-b1a1-574c4d25e9a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:04.426367 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fffce3-4a6a-45a2-b1a1-574c4d25e9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:04.427335 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fffce3-4a6a-45a2-b1a1-574c4d25e9a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:04.428273 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fffce3-4a6a-45a2-b1a1-574c4d25e9a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:04.429844 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fffce3-4a6a-45a2-b1a1-574c4d25e9a6 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:04.430050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c485e-e89f-4d56-b854-d9c4abde233c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:04.430853 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c485e-e89f-4d56-b854-d9c4abde233c admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:04.431309 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c485e-e89f-4d56-b854-d9c4abde233c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:04.431527 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c485e-e89f-4d56-b854-d9c4abde233c admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:04.432431 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649c485e-e89f-4d56-b854-d9c4abde233c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:04.435064 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649c485e-e89f-4d56-b854-d9c4abde233c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:04.443221 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 990/1982] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:04.444721 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fffce3-4a6a-45a2-b1a1-574c4d25e9a6 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:04.456220 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fffce3-4a6a-45a2-b1a1-574c4d25e9a6 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:04.461612 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645f410-8443-415f-9c11-ddc5aec47d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:04.462530 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645f410-8443-415f-9c11-ddc5aec47d14 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:04.462812 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645f410-8443-415f-9c11-ddc5aec47d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:04.463115 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645f410-8443-415f-9c11-ddc5aec47d14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:04.466240 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c645f410-8443-415f-9c11-ddc5aec47d14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:04.466417 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 993/1983] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:04 2022] POST /identity/v3/users => generated 286 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:04.466517 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c645f410-8443-415f-9c11-ddc5aec47d14 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:04.466605 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c485e-e89f-4d56-b854-d9c4abde233c admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:04.476874 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c485e-e89f-4d56-b854-d9c4abde233c admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:04.480676 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 991/1984] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:04.480676 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645f410-8443-415f-9c11-ddc5aec47d14 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:04.484238 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1befaaf-bef0-4a7b-b023-8368690a8b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:04.485049 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1befaaf-bef0-4a7b-b023-8368690a8b43 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:04.485412 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1befaaf-bef0-4a7b-b023-8368690a8b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:04.485762 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1befaaf-bef0-4a7b-b023-8368690a8b43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:04.486581 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1befaaf-bef0-4a7b-b023-8368690a8b43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:04.488520 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1befaaf-bef0-4a7b-b023-8368690a8b43 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:04.495406 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645f410-8443-415f-9c11-ddc5aec47d14 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:04.501194 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3532271a-3b2b-4938-ba28-dd69f4fdd7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:04.501548 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3532271a-3b2b-4938-ba28-dd69f4fdd7da admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:04.501824 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 994/1985] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:04.501933 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3532271a-3b2b-4938-ba28-dd69f4fdd7da admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:04.502105 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3532271a-3b2b-4938-ba28-dd69f4fdd7da admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:04.502886 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3532271a-3b2b-4938-ba28-dd69f4fdd7da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:04.503007 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1befaaf-bef0-4a7b-b023-8368690a8b43 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:04.504719 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3532271a-3b2b-4938-ba28-dd69f4fdd7da admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:04.513905 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1befaaf-bef0-4a7b-b023-8368690a8b43 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:04.516949 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 992/1986] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:04.517997 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3532271a-3b2b-4938-ba28-dd69f4fdd7da admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:04.523907 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09324109-d29d-4841-a563-60a14902d0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:04.524890 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09324109-d29d-4841-a563-60a14902d0b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:04.525268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09324109-d29d-4841-a563-60a14902d0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:04.525443 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09324109-d29d-4841-a563-60a14902d0b7 admin admin] PATH_INFO: `/v3/projects/02208a9da15e466da55d6179a72a9616/users/151c7b5d8ec64bbba4d2a7188a6a3541/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:04.531769 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3532271a-3b2b-4938-ba28-dd69f4fdd7da admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:04.537200 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae9b9d-881d-4a23-89ab-28d59f8bd394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:04.538399 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae9b9d-881d-4a23-89ab-28d59f8bd394 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:04.538810 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae9b9d-881d-4a23-89ab-28d59f8bd394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:04.539353 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae9b9d-881d-4a23-89ab-28d59f8bd394 admin admin] PATH_INFO: `/v3/projects/37efa7b6b18f4251a35d2415cad815ef/users/4a125018aa6749b2856f36a40f0ff040/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:04.542547 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09324109-d29d-4841-a563-60a14902d0b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02208a9da15e466da55d6179a72a9616, user_id=151c7b5d8ec64bbba4d2a7188a6a3541, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:04.545243 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09324109-d29d-4841-a563-60a14902d0b7 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:04.555724 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:04.562168 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 995/1987] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:04 2022] PUT /identity/v3/projects/02208a9da15e466da55d6179a72a9616/users/151c7b5d8ec64bbba4d2a7188a6a3541/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:04.563371 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09324109-d29d-4841-a563-60a14902d0b7 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:04.566456 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbae9b9d-881d-4a23-89ab-28d59f8bd394 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37efa7b6b18f4251a35d2415cad815ef, user_id=4a125018aa6749b2856f36a40f0ff040, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:04.569921 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbae9b9d-881d-4a23-89ab-28d59f8bd394 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:04.588627 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 993/1988] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:04 2022] PUT /identity/v3/projects/37efa7b6b18f4251a35d2415cad815ef/users/4a125018aa6749b2856f36a40f0ff040/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:04.596721 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2bd68-28f5-4728-b814-9dcfa2531b94 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:04.597088 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2bd68-28f5-4728-b814-9dcfa2531b94 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:04.597590 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2bd68-28f5-4728-b814-9dcfa2531b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:04.625097 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-9ed2bd68-28f5-4728-b814-9dcfa2531b94 None None] MFA Rules not processed for user `151c7b5d8ec64bbba4d2a7188a6a3541`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:04.626329 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-9ed2bd68-28f5-4728-b814-9dcfa2531b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:04.645374 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09324109-d29d-4841-a563-60a14902d0b7 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:04.697262 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 994/1989] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:04.698570 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518f445-48f2-488b-9f34-70399b2bf3d6 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:04.698859 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518f445-48f2-488b-9f34-70399b2bf3d6 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:04.699173 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518f445-48f2-488b-9f34-70399b2bf3d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:04.710187 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c15133-0f74-4d2d-8d24-fbdeab337818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:04.710885 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c15133-0f74-4d2d-8d24-fbdeab337818 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:04.711218 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c15133-0f74-4d2d-8d24-fbdeab337818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:04.711502 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c15133-0f74-4d2d-8d24-fbdeab337818 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:04.712272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c15133-0f74-4d2d-8d24-fbdeab337818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:04.713896 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c15133-0f74-4d2d-8d24-fbdeab337818 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:04.725200 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 996/1990] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:04.726367 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c15133-0f74-4d2d-8d24-fbdeab337818 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:04.730954 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-2518f445-48f2-488b-9f34-70399b2bf3d6 None None] MFA Rules not processed for user `4a125018aa6749b2856f36a40f0ff040`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:04.732193 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-2518f445-48f2-488b-9f34-70399b2bf3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:04.740255 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c15133-0f74-4d2d-8d24-fbdeab337818 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:04.745127 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91958a-ba33-4868-8a73-93f686086987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:04.745791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91958a-ba33-4868-8a73-93f686086987 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:04.746075 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91958a-ba33-4868-8a73-93f686086987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:04.746348 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91958a-ba33-4868-8a73-93f686086987 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:04.747065 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f91958a-ba33-4868-8a73-93f686086987 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:04.748975 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f91958a-ba33-4868-8a73-93f686086987 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:04.780624 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 997/1991] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:04 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:04.781459 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91958a-ba33-4868-8a73-93f686086987 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:04.785287 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:04.815112 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 995/1992] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:04.815958 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518f445-48f2-488b-9f34-70399b2bf3d6 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:04.826353 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518f445-48f2-488b-9f34-70399b2bf3d6 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:04.850558 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91958a-ba33-4868-8a73-93f686086987 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:04.855403 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395977d-8cf7-4aa6-a266-a76f1f28358f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:04.856365 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395977d-8cf7-4aa6-a266-a76f1f28358f admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:04.856727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395977d-8cf7-4aa6-a266-a76f1f28358f admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:04.857076 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395977d-8cf7-4aa6-a266-a76f1f28358f admin admin] PATH_INFO: `/v3/projects/287f6a351568438b995abe486e9c7b90/users/36a4b08e195c47eb9993f512532c4831/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:04.872791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b395977d-8cf7-4aa6-a266-a76f1f28358f admin admin] RBAC: Authorizing `identity:create_grant(project_id=287f6a351568438b995abe486e9c7b90, user_id=36a4b08e195c47eb9993f512532c4831, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:04.875440 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b395977d-8cf7-4aa6-a266-a76f1f28358f admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:04.885819 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f1ac7-3f29-4c99-9415-0b625997cd5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:04.886505 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f1ac7-3f29-4c99-9415-0b625997cd5a admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:04.886797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f1ac7-3f29-4c99-9415-0b625997cd5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:04.887107 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f1ac7-3f29-4c99-9415-0b625997cd5a admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:04.887833 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159f1ac7-3f29-4c99-9415-0b625997cd5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:04.888894 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 998/1993] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:04 2022] PUT /identity/v3/projects/287f6a351568438b995abe486e9c7b90/users/36a4b08e195c47eb9993f512532c4831/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:04.889679 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395977d-8cf7-4aa6-a266-a76f1f28358f admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:04.889770 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159f1ac7-3f29-4c99-9415-0b625997cd5a admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:04.893737 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:04.913311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 996/1994] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:04 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:04.914607 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506549f-e665-4e1e-a8fd-699636944ede None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:04.914895 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506549f-e665-4e1e-a8fd-699636944ede None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:04.915198 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506549f-e665-4e1e-a8fd-699636944ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:04.941932 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-5506549f-e665-4e1e-a8fd-699636944ede None None] MFA Rules not processed for user `36a4b08e195c47eb9993f512532c4831`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:04.945327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-5506549f-e665-4e1e-a8fd-699636944ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:04.963790 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395977d-8cf7-4aa6-a266-a76f1f28358f admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:04.969515 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f356408e-76d3-46a9-a591-b4d822efe2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:04.970185 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356408e-76d3-46a9-a591-b4d822efe2d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:04.970471 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356408e-76d3-46a9-a591-b4d822efe2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:04.970752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356408e-76d3-46a9-a591-b4d822efe2d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:04.971489 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f356408e-76d3-46a9-a591-b4d822efe2d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:04.974538 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f356408e-76d3-46a9-a591-b4d822efe2d2 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:05.004885 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 999/1995] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:04 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:05.004885 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f356408e-76d3-46a9-a591-b4d822efe2d2 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:05.024480 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 997/1996] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:05.024480 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506549f-e665-4e1e-a8fd-699636944ede None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:05.036838 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506549f-e665-4e1e-a8fd-699636944ede None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:05.072972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f356408e-76d3-46a9-a591-b4d822efe2d2 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:05.085849 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1faf340-7536-4594-ba08-0a00966da913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:05.086730 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1faf340-7536-4594-ba08-0a00966da913 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:05.087123 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1faf340-7536-4594-ba08-0a00966da913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:05.088240 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1faf340-7536-4594-ba08-0a00966da913 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:05.088413 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1faf340-7536-4594-ba08-0a00966da913 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:05.090101 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1faf340-7536-4594-ba08-0a00966da913 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:05.101507 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1000/1997] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:05.103921 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1faf340-7536-4594-ba08-0a00966da913 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:05.106532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c1faf340-7536-4594-ba08-0a00966da913 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:05.110514 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f672b6-74fa-4b13-8ae4-06f3938b7852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:05.111224 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f672b6-74fa-4b13-8ae4-06f3938b7852 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:05.111508 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f672b6-74fa-4b13-8ae4-06f3938b7852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:05.111827 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f672b6-74fa-4b13-8ae4-06f3938b7852 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:05.112512 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f672b6-74fa-4b13-8ae4-06f3938b7852 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:05.114461 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f672b6-74fa-4b13-8ae4-06f3938b7852 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:05.124678 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:05.147773 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 998/1998] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:05 2022] POST /identity/v3/users => generated 293 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:05.148141 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f672b6-74fa-4b13-8ae4-06f3938b7852 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:05.158610 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f672b6-74fa-4b13-8ae4-06f3938b7852 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:05.164278 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428bbca2-8cdc-4ce7-9825-e7fe7aba416e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:05.165281 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bbca2-8cdc-4ce7-9825-e7fe7aba416e admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:05.165565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bbca2-8cdc-4ce7-9825-e7fe7aba416e admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:05.165841 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bbca2-8cdc-4ce7-9825-e7fe7aba416e admin admin] PATH_INFO: `/v3/projects/bd69a60f677b4a1996790933f9197321/users/d46f29be3efb447896e2856f4132da1e/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:05.173394 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1faf340-7536-4594-ba08-0a00966da913 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:05.177791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e2bde-715c-4d7d-a748-9d1ede2d50ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:05.178609 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e2bde-715c-4d7d-a748-9d1ede2d50ea service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:05.178743 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e2bde-715c-4d7d-a748-9d1ede2d50ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:05.179357 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e2bde-715c-4d7d-a748-9d1ede2d50ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:05.183511 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428bbca2-8cdc-4ce7-9825-e7fe7aba416e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd69a60f677b4a1996790933f9197321, user_id=d46f29be3efb447896e2856f4132da1e, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:05.186293 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428bbca2-8cdc-4ce7-9825-e7fe7aba416e admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:05.206289 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 999/1999] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:05 2022] PUT /identity/v3/projects/bd69a60f677b4a1996790933f9197321/users/d46f29be3efb447896e2856f4132da1e/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:05.207354 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428bbca2-8cdc-4ce7-9825-e7fe7aba416e admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:05.229087 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:05.251772 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e2bde-715c-4d7d-a748-9d1ede2d50ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:05.253311 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e2bde-715c-4d7d-a748-9d1ede2d50ea service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:05.263504 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1001/2000] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:05 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:05.264634 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e2bde-715c-4d7d-a748-9d1ede2d50ea service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:05.279965 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e2bde-715c-4d7d-a748-9d1ede2d50ea service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:05.293310 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428bbca2-8cdc-4ce7-9825-e7fe7aba416e admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:05.298539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e2b9c1-9b43-48b2-9c1e-b32c966d59dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:05.299529 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2b9c1-9b43-48b2-9c1e-b32c966d59dc admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:05.299529 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2b9c1-9b43-48b2-9c1e-b32c966d59dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:05.299774 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2b9c1-9b43-48b2-9c1e-b32c966d59dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:05.300509 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e2b9c1-9b43-48b2-9c1e-b32c966d59dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:05.303675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e2b9c1-9b43-48b2-9c1e-b32c966d59dc admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:05.318686 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1000/2001] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:05.319639 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e2b9c1-9b43-48b2-9c1e-b32c966d59dc admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:05.332085 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e2b9c1-9b43-48b2-9c1e-b32c966d59dc admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:05.337106 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cbda95-90b4-4d3f-9416-5f7eb9716135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:05.337975 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cbda95-90b4-4d3f-9416-5f7eb9716135 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:05.338263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cbda95-90b4-4d3f-9416-5f7eb9716135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:05.338629 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cbda95-90b4-4d3f-9416-5f7eb9716135 admin admin] PATH_INFO: `/v3/users/36a4b08e195c47eb9993f512532c4831` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:05.339952 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cbda95-90b4-4d3f-9416-5f7eb9716135 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36a4b08e195c47eb9993f512532c4831)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:05.342026 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cbda95-90b4-4d3f-9416-5f7eb9716135 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:05.364696 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f12d15-54e3-4d17-a485-3116dbab4ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:05.365271 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f12d15-54e3-4d17-a485-3116dbab4ed0 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:05.365557 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f12d15-54e3-4d17-a485-3116dbab4ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:05.366077 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f12d15-54e3-4d17-a485-3116dbab4ed0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:05.366886 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f12d15-54e3-4d17-a485-3116dbab4ed0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:05.369383 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f12d15-54e3-4d17-a485-3116dbab4ed0 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:05.381555 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1002/2002] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:05.382381 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f12d15-54e3-4d17-a485-3116dbab4ed0 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:05.393969 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f12d15-54e3-4d17-a485-3116dbab4ed0 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:05.399573 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb0aa86-ca49-4b1e-8f30-1f16a5ff3115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:05.400462 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb0aa86-ca49-4b1e-8f30-1f16a5ff3115 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:05.400745 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb0aa86-ca49-4b1e-8f30-1f16a5ff3115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:05.401022 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb0aa86-ca49-4b1e-8f30-1f16a5ff3115 admin admin] PATH_INFO: `/v3/projects/6af0eaf232ca4198bc35669ac62acb6c/users/c123eaa22ccf431cb632d0209931b600/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:05.414240 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-40cbda95-90b4-4d3f-9416-5f7eb9716135 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36a4b08e195c47eb9993f512532c4831'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:05.415704 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-40cbda95-90b4-4d3f-9416-5f7eb9716135 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36a4b08e195c47eb9993f512532c4831'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:05.419647 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb0aa86-ca49-4b1e-8f30-1f16a5ff3115 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6af0eaf232ca4198bc35669ac62acb6c, user_id=c123eaa22ccf431cb632d0209931b600, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:05.422700 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb0aa86-ca49-4b1e-8f30-1f16a5ff3115 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:05.433875 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-40cbda95-90b4-4d3f-9416-5f7eb9716135 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36a4b08e195c47eb9993f512532c4831'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:05.444743 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1003/2003] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:05 2022] PUT /identity/v3/projects/6af0eaf232ca4198bc35669ac62acb6c/users/c123eaa22ccf431cb632d0209931b600/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:05.445548 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-40cbda95-90b4-4d3f-9416-5f7eb9716135 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36a4b08e195c47eb9993f512532c4831'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:05.445921 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb0aa86-ca49-4b1e-8f30-1f16a5ff3115 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:05.448033 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-8fb0aa86-ca49-4b1e-8f30-1f16a5ff3115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:05.455290 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-40cbda95-90b4-4d3f-9416-5f7eb9716135 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36a4b08e195c47eb9993f512532c4831'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:05.464320 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1001/2004] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:05 2022] DELETE /identity/v3/users/36a4b08e195c47eb9993f512532c4831 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:05.467766 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cbda95-90b4-4d3f-9416-5f7eb9716135 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:05.467766 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-40cbda95-90b4-4d3f-9416-5f7eb9716135 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:05.528593 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb0aa86-ca49-4b1e-8f30-1f16a5ff3115 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:05.532418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:05.563471 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cbda95-90b4-4d3f-9416-5f7eb9716135 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:05.569948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd98b038-c584-4d8a-935b-adb0eba66f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:05.570774 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98b038-c584-4d8a-935b-adb0eba66f6a admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:05.571189 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98b038-c584-4d8a-935b-adb0eba66f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:05.571560 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98b038-c584-4d8a-935b-adb0eba66f6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:05.572405 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd98b038-c584-4d8a-935b-adb0eba66f6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:05.574136 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd98b038-c584-4d8a-935b-adb0eba66f6a admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:05.590232 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:05.593810 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1002/2005] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:05.595739 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460135b-9d41-46ad-b8d7-4d31b590f280 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:05.596363 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460135b-9d41-46ad-b8d7-4d31b590f280 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:05.597023 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460135b-9d41-46ad-b8d7-4d31b590f280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:05.600900 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b5d7a-a37d-4e7a-8c12-dfd298717f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:05.601972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b5d7a-a37d-4e7a-8c12-dfd298717f11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:05.602578 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b5d7a-a37d-4e7a-8c12-dfd298717f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:05.602927 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b5d7a-a37d-4e7a-8c12-dfd298717f11 admin admin] PATH_INFO: `/v3/projects/bd69a60f677b4a1996790933f9197321/users/d46f29be3efb447896e2856f4132da1e/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:05.605677 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8b5d7a-a37d-4e7a-8c12-dfd298717f11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd69a60f677b4a1996790933f9197321, user_id=d46f29be3efb447896e2856f4132da1e, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:05.608560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8b5d7a-a37d-4e7a-8c12-dfd298717f11 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:05.623587 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-d460135b-9d41-46ad-b8d7-4d31b590f280 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:05.625806 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d460135b-9d41-46ad-b8d7-4d31b590f280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:05.637697 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1004/2006] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:05 2022] PUT /identity/v3/projects/bd69a60f677b4a1996790933f9197321/users/d46f29be3efb447896e2856f4132da1e/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:05.638881 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b5d7a-a37d-4e7a-8c12-dfd298717f11 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:05.700014 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1003/2007] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:05.701737 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df5308-5f4f-4547-b86f-5ee17522a8ee None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:05.702104 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df5308-5f4f-4547-b86f-5ee17522a8ee None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:05.702391 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df5308-5f4f-4547-b86f-5ee17522a8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:05.730229 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-58df5308-5f4f-4547-b86f-5ee17522a8ee None None] MFA Rules not processed for user `d46f29be3efb447896e2856f4132da1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:05.731468 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-58df5308-5f4f-4547-b86f-5ee17522a8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:05.732735 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b5d7a-a37d-4e7a-8c12-dfd298717f11 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:05.737944 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f335b86-7d2d-40d2-8f5d-ff237c46a6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:05.738886 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f335b86-7d2d-40d2-8f5d-ff237c46a6a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:05.739273 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f335b86-7d2d-40d2-8f5d-ff237c46a6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:05.739876 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f335b86-7d2d-40d2-8f5d-ff237c46a6a0 admin admin] PATH_INFO: `/v3/projects/6af0eaf232ca4198bc35669ac62acb6c/users/c123eaa22ccf431cb632d0209931b600/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:05.742344 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f335b86-7d2d-40d2-8f5d-ff237c46a6a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6af0eaf232ca4198bc35669ac62acb6c, user_id=c123eaa22ccf431cb632d0209931b600, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:05.745138 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f335b86-7d2d-40d2-8f5d-ff237c46a6a0 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:05.762560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1005/2008] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:05 2022] PUT /identity/v3/projects/6af0eaf232ca4198bc35669ac62acb6c/users/c123eaa22ccf431cb632d0209931b600/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:05.763638 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f335b86-7d2d-40d2-8f5d-ff237c46a6a0 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:05.765232 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1f335b86-7d2d-40d2-8f5d-ff237c46a6a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:05.816686 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1004/2009] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:05 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:05.818025 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab18c2-515d-450b-a27e-1e2bc6994a71 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:05.818319 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab18c2-515d-450b-a27e-1e2bc6994a71 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:05.818602 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab18c2-515d-450b-a27e-1e2bc6994a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:05.836206 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f335b86-7d2d-40d2-8f5d-ff237c46a6a0 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:05.839955 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8c7fe-f392-4090-8eac-73abcc011d57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:05.840639 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8c7fe-f392-4090-8eac-73abcc011d57 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:05.840924 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8c7fe-f392-4090-8eac-73abcc011d57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:05.841202 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8c7fe-f392-4090-8eac-73abcc011d57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:05.845629 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-e4ab18c2-515d-450b-a27e-1e2bc6994a71 None None] MFA Rules not processed for user `c123eaa22ccf431cb632d0209931b600`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:05.847412 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e4ab18c2-515d-450b-a27e-1e2bc6994a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:05.898039 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:05.904485 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db8c7fe-f392-4090-8eac-73abcc011d57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:05.905952 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db8c7fe-f392-4090-8eac-73abcc011d57 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:05.916959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1006/2010] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:05.917948 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8c7fe-f392-4090-8eac-73abcc011d57 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:05.919381 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-3db8c7fe-f392-4090-8eac-73abcc011d57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:05.920493 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1005/2011] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:05 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:05.933106 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab18c2-515d-450b-a27e-1e2bc6994a71 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:05.934979 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e4ab18c2-515d-450b-a27e-1e2bc6994a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:05.962632 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:05.985660 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8c7fe-f392-4090-8eac-73abcc011d57 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:05.990265 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d8d633-fc08-4eba-9755-b5c39bb1d016 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:05.991190 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8d633-fc08-4eba-9755-b5c39bb1d016 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:05.991773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8d633-fc08-4eba-9755-b5c39bb1d016 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:05.991920 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8d633-fc08-4eba-9755-b5c39bb1d016 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:06.014245 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab18c2-515d-450b-a27e-1e2bc6994a71 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:06.024654 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b57b839-8f3f-4fdf-9434-540585128c37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:06.025435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57b839-8f3f-4fdf-9434-540585128c37 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:06.025971 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57b839-8f3f-4fdf-9434-540585128c37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:06.026269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57b839-8f3f-4fdf-9434-540585128c37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:06.039299 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b57b839-8f3f-4fdf-9434-540585128c37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:06.040916 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b57b839-8f3f-4fdf-9434-540585128c37 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:06.053038 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1006/2012] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:05 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:44:06.097947 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d8d633-fc08-4eba-9755-b5c39bb1d016 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:06.100060 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d8d633-fc08-4eba-9755-b5c39bb1d016 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:06.112966 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1007/2013] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:05 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:06.253686 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b57b839-8f3f-4fdf-9434-540585128c37 service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:06.256233 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1b57b839-8f3f-4fdf-9434-540585128c37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:06.287045 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b57b839-8f3f-4fdf-9434-540585128c37 service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:06.310820 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed08d8-4d92-4bfa-a707-577ccdad53eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:06.312357 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed08d8-4d92-4bfa-a707-577ccdad53eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:06.314008 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed08d8-4d92-4bfa-a707-577ccdad53eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:06.314738 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed08d8-4d92-4bfa-a707-577ccdad53eb admin admin] PATH_INFO: `/v3/projects/287f6a351568438b995abe486e9c7b90` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:06.318279 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ed08d8-4d92-4bfa-a707-577ccdad53eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=287f6a351568438b995abe486e9c7b90)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:06.321296 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ed08d8-4d92-4bfa-a707-577ccdad53eb admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:06.464062 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-81ed08d8-4d92-4bfa-a707-577ccdad53eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '287f6a351568438b995abe486e9c7b90', 'request_id': 'req-81ed08d8-4d92-4bfa-a707-577ccdad53eb'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:06.479521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-81ed08d8-4d92-4bfa-a707-577ccdad53eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '287f6a351568438b995abe486e9c7b90', 'request_id': 'req-81ed08d8-4d92-4bfa-a707-577ccdad53eb'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:06.488012 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-81ed08d8-4d92-4bfa-a707-577ccdad53eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '287f6a351568438b995abe486e9c7b90', 'request_id': 'req-81ed08d8-4d92-4bfa-a707-577ccdad53eb'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:06.503923 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-81ed08d8-4d92-4bfa-a707-577ccdad53eb admin admin] The token cache is being invalidate because project 287f6a351568438b995abe486e9c7b90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:06.504103 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-81ed08d8-4d92-4bfa-a707-577ccdad53eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:06.505523 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-81ed08d8-4d92-4bfa-a707-577ccdad53eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:06.512506 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17219fd-81a8-4bf4-9c9a-290e1a60d1f6 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:06.513223 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17219fd-81a8-4bf4-9c9a-290e1a60d1f6 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:06.513512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17219fd-81a8-4bf4-9c9a-290e1a60d1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:06.515343 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1007/2014] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:06 2022] DELETE /identity/v3/projects/287f6a351568438b995abe486e9c7b90 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:06.533988 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-d17219fd-81a8-4bf4-9c9a-290e1a60d1f6 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:06.535303 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d17219fd-81a8-4bf4-9c9a-290e1a60d1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:06.635867 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1008/2015] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:06.643805 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb602ab-45c3-495e-8ab6-fed7ba85219e None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:06.644316 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb602ab-45c3-495e-8ab6-fed7ba85219e None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:06.644410 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb602ab-45c3-495e-8ab6-fed7ba85219e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:06.674705 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-deb602ab-45c3-495e-8ab6-fed7ba85219e None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:06.676214 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-deb602ab-45c3-495e-8ab6-fed7ba85219e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:06.690108 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1008/2016] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:06.697593 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17219fd-81a8-4bf4-9c9a-290e1a60d1f6 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:06.717052 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17219fd-81a8-4bf4-9c9a-290e1a60d1f6 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:06.728022 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6e6d8-be53-48c7-bb96-7a8e757b266c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:06.728783 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6e6d8-be53-48c7-bb96-7a8e757b266c admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:06.729087 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6e6d8-be53-48c7-bb96-7a8e757b266c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:06.729416 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6e6d8-be53-48c7-bb96-7a8e757b266c admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:06.730436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb6e6d8-be53-48c7-bb96-7a8e757b266c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:44:06.730520 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb6e6d8-be53-48c7-bb96-7a8e757b266c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:06.732256 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb6e6d8-be53-48c7-bb96-7a8e757b266c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:06.753359 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1009/2017] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:44:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:44:06.754149 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb602ab-45c3-495e-8ab6-fed7ba85219e None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:06.768878 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb602ab-45c3-495e-8ab6-fed7ba85219e None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:06.775764 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de088cc4-3eba-45e5-823c-4ec627fc4c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:06.776275 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de088cc4-3eba-45e5-823c-4ec627fc4c6f admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:06.776954 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de088cc4-3eba-45e5-823c-4ec627fc4c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:06.777377 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de088cc4-3eba-45e5-823c-4ec627fc4c6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:06.778789 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de088cc4-3eba-45e5-823c-4ec627fc4c6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:06.783355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de088cc4-3eba-45e5-823c-4ec627fc4c6f admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:06.786927 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6e6d8-be53-48c7-bb96-7a8e757b266c admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:06.790291 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-bcb6e6d8-be53-48c7-bb96-7a8e757b266c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:06.807144 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6e6d8-be53-48c7-bb96-7a8e757b266c admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:06.813989 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b044b-f722-48d0-824c-73ee3fc3950a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:06.814957 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b044b-f722-48d0-824c-73ee3fc3950a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:06.815281 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b044b-f722-48d0-824c-73ee3fc3950a admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:06.815851 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b044b-f722-48d0-824c-73ee3fc3950a admin admin] PATH_INFO: `/v3/users/4a125018aa6749b2856f36a40f0ff040` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:06.817878 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b044b-f722-48d0-824c-73ee3fc3950a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a125018aa6749b2856f36a40f0ff040)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:06.819981 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b044b-f722-48d0-824c-73ee3fc3950a admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:06.840326 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1009/2018] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:06 2022] POST /identity/v3/projects => generated 373 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:44:06.841469 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de088cc4-3eba-45e5-823c-4ec627fc4c6f admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:06.915966 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a34b044b-f722-48d0-824c-73ee3fc3950a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a125018aa6749b2856f36a40f0ff040'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:06.916694 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a34b044b-f722-48d0-824c-73ee3fc3950a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a125018aa6749b2856f36a40f0ff040'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:06.933413 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a34b044b-f722-48d0-824c-73ee3fc3950a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a125018aa6749b2856f36a40f0ff040'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:06.934153 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a34b044b-f722-48d0-824c-73ee3fc3950a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a125018aa6749b2856f36a40f0ff040'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:06.938592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:06.944293 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:06.946576 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a34b044b-f722-48d0-824c-73ee3fc3950a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a125018aa6749b2856f36a40f0ff040'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:06.964939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de088cc4-3eba-45e5-823c-4ec627fc4c6f admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:06.966898 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1010/2019] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:06 2022] DELETE /identity/v3/users/4a125018aa6749b2856f36a40f0ff040 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:06.967828 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b044b-f722-48d0-824c-73ee3fc3950a admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:06.969197 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a34b044b-f722-48d0-824c-73ee3fc3950a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:07.031564 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75142938-a446-4905-98ff-b18ca97126e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:07.032243 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75142938-a446-4905-98ff-b18ca97126e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:07.032529 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75142938-a446-4905-98ff-b18ca97126e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:07.032806 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75142938-a446-4905-98ff-b18ca97126e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:07.033492 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75142938-a446-4905-98ff-b18ca97126e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:07.036663 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75142938-a446-4905-98ff-b18ca97126e2 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:07.041094 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b044b-f722-48d0-824c-73ee3fc3950a admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:07.046085 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819149df-1f83-4888-bd69-246def1da6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:07.046646 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819149df-1f83-4888-bd69-246def1da6fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:07.046930 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819149df-1f83-4888-bd69-246def1da6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:07.047235 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819149df-1f83-4888-bd69-246def1da6fe admin admin] PATH_INFO: `/v3/users/151c7b5d8ec64bbba4d2a7188a6a3541` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:07.048440 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819149df-1f83-4888-bd69-246def1da6fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=151c7b5d8ec64bbba4d2a7188a6a3541)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:07.050361 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819149df-1f83-4888-bd69-246def1da6fe admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:07.057451 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1010/2020] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:06 2022] POST /identity/v3/users => generated 287 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:07.058723 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3908dc0-4411-407a-9a84-dc9344108f2b None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:07.059033 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3908dc0-4411-407a-9a84-dc9344108f2b None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:07.059320 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3908dc0-4411-407a-9a84-dc9344108f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:07.076505 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-e3908dc0-4411-407a-9a84-dc9344108f2b None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:07.077718 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e3908dc0-4411-407a-9a84-dc9344108f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:07.085698 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1011/2021] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:07.086479 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3908dc0-4411-407a-9a84-dc9344108f2b None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:07.087829 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e3908dc0-4411-407a-9a84-dc9344108f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:07.101153 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3908dc0-4411-407a-9a84-dc9344108f2b None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:07.108869 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7061895-474e-4eb8-9531-65d2763ced74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:07.108869 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7061895-474e-4eb8-9531-65d2763ced74 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:07.108869 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7061895-474e-4eb8-9531-65d2763ced74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:07.108869 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7061895-474e-4eb8-9531-65d2763ced74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:07.109913 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7061895-474e-4eb8-9531-65d2763ced74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:07.110344 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-819149df-1f83-4888-bd69-246def1da6fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '151c7b5d8ec64bbba4d2a7188a6a3541'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:07.111075 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-819149df-1f83-4888-bd69-246def1da6fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '151c7b5d8ec64bbba4d2a7188a6a3541'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:07.112880 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7061895-474e-4eb8-9531-65d2763ced74 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:07.124825 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-819149df-1f83-4888-bd69-246def1da6fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '151c7b5d8ec64bbba4d2a7188a6a3541'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:07.125635 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-819149df-1f83-4888-bd69-246def1da6fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '151c7b5d8ec64bbba4d2a7188a6a3541'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:07.127806 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1012/2022] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:07.128672 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7061895-474e-4eb8-9531-65d2763ced74 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:07.132518 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e7061895-474e-4eb8-9531-65d2763ced74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:07.141282 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-819149df-1f83-4888-bd69-246def1da6fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '151c7b5d8ec64bbba4d2a7188a6a3541'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:07.154803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1011/2023] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:06 2022] DELETE /identity/v3/users/151c7b5d8ec64bbba4d2a7188a6a3541 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:07.155673 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819149df-1f83-4888-bd69-246def1da6fe admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:07.157027 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-819149df-1f83-4888-bd69-246def1da6fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:07.196735 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7061895-474e-4eb8-9531-65d2763ced74 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:07.225446 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819149df-1f83-4888-bd69-246def1da6fe admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:07.230131 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc382944-4fce-4288-a900-30e76cce5c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:07.230804 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc382944-4fce-4288-a900-30e76cce5c33 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:07.231115 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc382944-4fce-4288-a900-30e76cce5c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:07.231397 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc382944-4fce-4288-a900-30e76cce5c33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:07.232138 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc382944-4fce-4288-a900-30e76cce5c33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:07.233766 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc382944-4fce-4288-a900-30e76cce5c33 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:07.242413 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:07.249477 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1012/2024] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:07.251411 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41115cc1-54a7-453a-a0a7-e64855256be5 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:07.251866 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41115cc1-54a7-453a-a0a7-e64855256be5 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:07.251957 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41115cc1-54a7-453a-a0a7-e64855256be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:07.255390 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6f1d6-e6b0-43ff-864e-45f2b39f4232 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:07.256319 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6f1d6-e6b0-43ff-864e-45f2b39f4232 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:07.256653 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6f1d6-e6b0-43ff-864e-45f2b39f4232 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:07.257119 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6f1d6-e6b0-43ff-864e-45f2b39f4232 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:07.259719 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-68c6f1d6-e6b0-43ff-864e-45f2b39f4232 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:07.273377 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c6f1d6-e6b0-43ff-864e-45f2b39f4232 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:07.273556 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-41115cc1-54a7-453a-a0a7-e64855256be5 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:07.274655 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-41115cc1-54a7-453a-a0a7-e64855256be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:07.274860 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c6f1d6-e6b0-43ff-864e-45f2b39f4232 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:07.283090 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1013/2025] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:07.285662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1013/2026] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:07.287996 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41115cc1-54a7-453a-a0a7-e64855256be5 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:07.297867 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41115cc1-54a7-453a-a0a7-e64855256be5 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:07.304020 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6f1d6-e6b0-43ff-864e-45f2b39f4232 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:07.304142 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404291de-69d1-4e72-b2f5-048f4487982b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:07.305045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404291de-69d1-4e72-b2f5-048f4487982b admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:07.305394 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404291de-69d1-4e72-b2f5-048f4487982b admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:07.305740 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404291de-69d1-4e72-b2f5-048f4487982b admin admin] PATH_INFO: `/v3/projects/77d56467f04249f6a419a0f843195965/users/789e349e3b1c41df8494572672541885/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:07.306728 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-68c6f1d6-e6b0-43ff-864e-45f2b39f4232 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:07.319620 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6f1d6-e6b0-43ff-864e-45f2b39f4232 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:07.325502 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404291de-69d1-4e72-b2f5-048f4487982b admin admin] RBAC: Authorizing `identity:create_grant(project_id=77d56467f04249f6a419a0f843195965, user_id=789e349e3b1c41df8494572672541885, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:07.327950 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913274c5-8497-4b18-aafe-451107ebfe4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:07.328693 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913274c5-8497-4b18-aafe-451107ebfe4b service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:07.329012 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913274c5-8497-4b18-aafe-451107ebfe4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:07.329293 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913274c5-8497-4b18-aafe-451107ebfe4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:07.329413 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404291de-69d1-4e72-b2f5-048f4487982b admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:07.342323 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913274c5-8497-4b18-aafe-451107ebfe4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:07.344565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913274c5-8497-4b18-aafe-451107ebfe4b service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:07.351405 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1014/2027] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:07 2022] PUT /identity/v3/projects/77d56467f04249f6a419a0f843195965/users/789e349e3b1c41df8494572672541885/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:07.355770 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1014/2028] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:44:07.365584 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff091d-f1a6-4f03-8a69-74224ee298aa None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:07.365943 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff091d-f1a6-4f03-8a69-74224ee298aa None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:07.366539 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff091d-f1a6-4f03-8a69-74224ee298aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:07.399503 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-29ff091d-f1a6-4f03-8a69-74224ee298aa None None] MFA Rules not processed for user `789e349e3b1c41df8494572672541885`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:07.400788 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-29ff091d-f1a6-4f03-8a69-74224ee298aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:07.499243 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1015/2029] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:07.511473 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913274c5-8497-4b18-aafe-451107ebfe4b service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:07.513033 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-913274c5-8497-4b18-aafe-451107ebfe4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:07.527948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:07.587623 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913274c5-8497-4b18-aafe-451107ebfe4b service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:07.704570 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578c17a-02a8-413d-b9f2-0100b5692ba7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:07.705245 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578c17a-02a8-413d-b9f2-0100b5692ba7 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:07.705535 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578c17a-02a8-413d-b9f2-0100b5692ba7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:07.705816 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578c17a-02a8-413d-b9f2-0100b5692ba7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:07.709112 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff091d-f1a6-4f03-8a69-74224ee298aa None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:07.710597 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-29ff091d-f1a6-4f03-8a69-74224ee298aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:07.733771 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f578c17a-02a8-413d-b9f2-0100b5692ba7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:07.739431 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f578c17a-02a8-413d-b9f2-0100b5692ba7 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:07.762097 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1015/2030] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:07.826050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff091d-f1a6-4f03-8a69-74224ee298aa None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:07.834554 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd12ed-1508-4946-9c4c-5f6b9fec9889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:07.835462 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd12ed-1508-4946-9c4c-5f6b9fec9889 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:07.835879 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd12ed-1508-4946-9c4c-5f6b9fec9889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:07.836164 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd12ed-1508-4946-9c4c-5f6b9fec9889 admin admin] PATH_INFO: `/v3/projects/37efa7b6b18f4251a35d2415cad815ef` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:07.837590 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd12ed-1508-4946-9c4c-5f6b9fec9889 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37efa7b6b18f4251a35d2415cad815ef)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:07.839767 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd12ed-1508-4946-9c4c-5f6b9fec9889 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:07.890128 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578c17a-02a8-413d-b9f2-0100b5692ba7 service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:07.891875 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f578c17a-02a8-413d-b9f2-0100b5692ba7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:07.908882 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578c17a-02a8-413d-b9f2-0100b5692ba7 service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:07.926583 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f8cd12ed-1508-4946-9c4c-5f6b9fec9889 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37efa7b6b18f4251a35d2415cad815ef', 'request_id': 'req-f8cd12ed-1508-4946-9c4c-5f6b9fec9889'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:07.938482 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f8cd12ed-1508-4946-9c4c-5f6b9fec9889 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37efa7b6b18f4251a35d2415cad815ef', 'request_id': 'req-f8cd12ed-1508-4946-9c4c-5f6b9fec9889'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:07.946873 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f8cd12ed-1508-4946-9c4c-5f6b9fec9889 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37efa7b6b18f4251a35d2415cad815ef', 'request_id': 'req-f8cd12ed-1508-4946-9c4c-5f6b9fec9889'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:07.949351 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:07.961246 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f8cd12ed-1508-4946-9c4c-5f6b9fec9889 admin admin] The token cache is being invalidate because project 37efa7b6b18f4251a35d2415cad815ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:07.961500 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f8cd12ed-1508-4946-9c4c-5f6b9fec9889 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:07.963246 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f8cd12ed-1508-4946-9c4c-5f6b9fec9889 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:07.965507 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1016/2031] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:07 2022] DELETE /identity/v3/projects/37efa7b6b18f4251a35d2415cad815ef => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:07.969250 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd12ed-1508-4946-9c4c-5f6b9fec9889 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:07.970972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f8cd12ed-1508-4946-9c4c-5f6b9fec9889 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:08.004631 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d2448-dc02-4a3a-ae11-7e6df1b683ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:08.006069 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d2448-dc02-4a3a-ae11-7e6df1b683ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:08.007398 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d2448-dc02-4a3a-ae11-7e6df1b683ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:08.008048 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d2448-dc02-4a3a-ae11-7e6df1b683ce admin admin] PATH_INFO: `/v3/projects/02208a9da15e466da55d6179a72a9616` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:08.009583 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d2448-dc02-4a3a-ae11-7e6df1b683ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=02208a9da15e466da55d6179a72a9616)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:08.012217 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d2448-dc02-4a3a-ae11-7e6df1b683ce admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:08.068691 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd12ed-1508-4946-9c4c-5f6b9fec9889 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:08.075490 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80cf162-15d9-4983-a9d3-3d79b19e2af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:08.076386 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80cf162-15d9-4983-a9d3-3d79b19e2af8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:08.076682 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80cf162-15d9-4983-a9d3-3d79b19e2af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:08.076942 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80cf162-15d9-4983-a9d3-3d79b19e2af8 admin admin] PATH_INFO: `/v3/users/c123eaa22ccf431cb632d0209931b600` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:08.078882 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80cf162-15d9-4983-a9d3-3d79b19e2af8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c123eaa22ccf431cb632d0209931b600)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:08.083447 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80cf162-15d9-4983-a9d3-3d79b19e2af8 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:08.110726 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-316d2448-dc02-4a3a-ae11-7e6df1b683ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02208a9da15e466da55d6179a72a9616', 'request_id': 'req-316d2448-dc02-4a3a-ae11-7e6df1b683ce'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:08.133618 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-316d2448-dc02-4a3a-ae11-7e6df1b683ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02208a9da15e466da55d6179a72a9616', 'request_id': 'req-316d2448-dc02-4a3a-ae11-7e6df1b683ce'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:08.159945 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-316d2448-dc02-4a3a-ae11-7e6df1b683ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02208a9da15e466da55d6179a72a9616', 'request_id': 'req-316d2448-dc02-4a3a-ae11-7e6df1b683ce'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:08.169847 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:08.179652 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-316d2448-dc02-4a3a-ae11-7e6df1b683ce admin admin] The token cache is being invalidate because project 02208a9da15e466da55d6179a72a9616 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:08.180226 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-316d2448-dc02-4a3a-ae11-7e6df1b683ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:08.182584 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-316d2448-dc02-4a3a-ae11-7e6df1b683ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:08.185703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1016/2032] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:07 2022] DELETE /identity/v3/projects/02208a9da15e466da55d6179a72a9616 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:08.189488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d2448-dc02-4a3a-ae11-7e6df1b683ce admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:08.191648 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-316d2448-dc02-4a3a-ae11-7e6df1b683ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:08.213988 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f80cf162-15d9-4983-a9d3-3d79b19e2af8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c123eaa22ccf431cb632d0209931b600'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:08.215433 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f80cf162-15d9-4983-a9d3-3d79b19e2af8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c123eaa22ccf431cb632d0209931b600'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:08.232314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f80cf162-15d9-4983-a9d3-3d79b19e2af8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c123eaa22ccf431cb632d0209931b600'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:08.233340 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f80cf162-15d9-4983-a9d3-3d79b19e2af8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c123eaa22ccf431cb632d0209931b600'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:08.245542 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f80cf162-15d9-4983-a9d3-3d79b19e2af8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c123eaa22ccf431cb632d0209931b600'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:08.262284 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1017/2033] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:07 2022] DELETE /identity/v3/users/c123eaa22ccf431cb632d0209931b600 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:08.291796 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d2448-dc02-4a3a-ae11-7e6df1b683ce admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:08.354376 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da87f96-a23f-47e2-bcf9-45509540ab5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:08.355495 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da87f96-a23f-47e2-bcf9-45509540ab5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:08.355777 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da87f96-a23f-47e2-bcf9-45509540ab5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:08.356049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da87f96-a23f-47e2-bcf9-45509540ab5a admin admin] PATH_INFO: `/v3/users/d46f29be3efb447896e2856f4132da1e` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:08.357273 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da87f96-a23f-47e2-bcf9-45509540ab5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d46f29be3efb447896e2856f4132da1e)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:08.359226 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da87f96-a23f-47e2-bcf9-45509540ab5a admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:08.408233 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-2da87f96-a23f-47e2-bcf9-45509540ab5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd46f29be3efb447896e2856f4132da1e'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:08.408925 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-2da87f96-a23f-47e2-bcf9-45509540ab5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd46f29be3efb447896e2856f4132da1e'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:08.425509 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-2da87f96-a23f-47e2-bcf9-45509540ab5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd46f29be3efb447896e2856f4132da1e'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:08.433508 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-2da87f96-a23f-47e2-bcf9-45509540ab5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd46f29be3efb447896e2856f4132da1e'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:08.441829 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-2da87f96-a23f-47e2-bcf9-45509540ab5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd46f29be3efb447896e2856f4132da1e'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:08.448491 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1017/2034] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:08 2022] DELETE /identity/v3/users/d46f29be3efb447896e2856f4132da1e => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:08.555665 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80cf162-15d9-4983-a9d3-3d79b19e2af8 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:08.556990 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f80cf162-15d9-4983-a9d3-3d79b19e2af8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:08.574001 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80cf162-15d9-4983-a9d3-3d79b19e2af8 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:08.650977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3731e6f7-8e3a-4b01-8fe6-7edfc93096bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:08.652070 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731e6f7-8e3a-4b01-8fe6-7edfc93096bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:08.652419 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731e6f7-8e3a-4b01-8fe6-7edfc93096bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:08.652827 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731e6f7-8e3a-4b01-8fe6-7edfc93096bf admin admin] PATH_INFO: `/v3/projects/6af0eaf232ca4198bc35669ac62acb6c` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:08.654142 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3731e6f7-8e3a-4b01-8fe6-7edfc93096bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=6af0eaf232ca4198bc35669ac62acb6c)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:08.656248 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3731e6f7-8e3a-4b01-8fe6-7edfc93096bf admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:08.670744 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:08.719411 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3731e6f7-8e3a-4b01-8fe6-7edfc93096bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6af0eaf232ca4198bc35669ac62acb6c', 'request_id': 'req-3731e6f7-8e3a-4b01-8fe6-7edfc93096bf'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:08.732069 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3731e6f7-8e3a-4b01-8fe6-7edfc93096bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6af0eaf232ca4198bc35669ac62acb6c', 'request_id': 'req-3731e6f7-8e3a-4b01-8fe6-7edfc93096bf'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:08.739227 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3731e6f7-8e3a-4b01-8fe6-7edfc93096bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6af0eaf232ca4198bc35669ac62acb6c', 'request_id': 'req-3731e6f7-8e3a-4b01-8fe6-7edfc93096bf'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:08.752259 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3731e6f7-8e3a-4b01-8fe6-7edfc93096bf admin admin] The token cache is being invalidate because project 6af0eaf232ca4198bc35669ac62acb6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:08.752556 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3731e6f7-8e3a-4b01-8fe6-7edfc93096bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:08.753258 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3731e6f7-8e3a-4b01-8fe6-7edfc93096bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:08.755485 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1018/2035] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:08 2022] DELETE /identity/v3/projects/6af0eaf232ca4198bc35669ac62acb6c => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:08.760438 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70f0f5-05c8-4401-9885-6630276883c5 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:08.760910 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70f0f5-05c8-4401-9885-6630276883c5 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:08.761022 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70f0f5-05c8-4401-9885-6630276883c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:08.781657 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-1d70f0f5-05c8-4401-9885-6630276883c5 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:08.782883 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1d70f0f5-05c8-4401-9885-6630276883c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:08.797345 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3731e6f7-8e3a-4b01-8fe6-7edfc93096bf admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:08.799616 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-3731e6f7-8e3a-4b01-8fe6-7edfc93096bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:08.809514 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:08.883032 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1018/2036] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:08.889200 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae4faf-0283-4076-bf8d-b3cea7f6b193 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:08.889711 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae4faf-0283-4076-bf8d-b3cea7f6b193 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:08.890142 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae4faf-0283-4076-bf8d-b3cea7f6b193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:08.920968 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-90ae4faf-0283-4076-bf8d-b3cea7f6b193 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:08.920968 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-90ae4faf-0283-4076-bf8d-b3cea7f6b193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:08.921724 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3731e6f7-8e3a-4b01-8fe6-7edfc93096bf admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:08.925050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1019/2037] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:44:08.930247 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb0945-d120-4557-a3f8-65c2dc59e578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:08.930247 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb0945-d120-4557-a3f8-65c2dc59e578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:08.930247 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb0945-d120-4557-a3f8-65c2dc59e578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:08.930247 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb0945-d120-4557-a3f8-65c2dc59e578 admin admin] PATH_INFO: `/v3/projects/bd69a60f677b4a1996790933f9197321` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:08.930565 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae4faf-0283-4076-bf8d-b3cea7f6b193 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:08.930636 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb0945-d120-4557-a3f8-65c2dc59e578 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd69a60f677b4a1996790933f9197321)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:08.931940 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb0945-d120-4557-a3f8-65c2dc59e578 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:08.940203 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae4faf-0283-4076-bf8d-b3cea7f6b193 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:08.949606 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898cbe86-80e6-4fed-813b-6a853f3a785f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:08.950337 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898cbe86-80e6-4fed-813b-6a853f3a785f admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:08.950642 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898cbe86-80e6-4fed-813b-6a853f3a785f admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:08.950969 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898cbe86-80e6-4fed-813b-6a853f3a785f admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:08.951935 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898cbe86-80e6-4fed-813b-6a853f3a785f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:44:08.952253 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898cbe86-80e6-4fed-813b-6a853f3a785f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:08.954442 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898cbe86-80e6-4fed-813b-6a853f3a785f admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:08.975347 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1020/2038] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:44:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:44:08.979423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898cbe86-80e6-4fed-813b-6a853f3a785f admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:08.997931 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898cbe86-80e6-4fed-813b-6a853f3a785f admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:09.016036 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-bbbb0945-d120-4557-a3f8-65c2dc59e578 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd69a60f677b4a1996790933f9197321', 'request_id': 'req-bbbb0945-d120-4557-a3f8-65c2dc59e578'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:09.031383 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-bbbb0945-d120-4557-a3f8-65c2dc59e578 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd69a60f677b4a1996790933f9197321', 'request_id': 'req-bbbb0945-d120-4557-a3f8-65c2dc59e578'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:09.045507 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-bbbb0945-d120-4557-a3f8-65c2dc59e578 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd69a60f677b4a1996790933f9197321', 'request_id': 'req-bbbb0945-d120-4557-a3f8-65c2dc59e578'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:09.060577 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-bbbb0945-d120-4557-a3f8-65c2dc59e578 admin admin] The token cache is being invalidate because project bd69a60f677b4a1996790933f9197321 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:09.060880 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-bbbb0945-d120-4557-a3f8-65c2dc59e578 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:09.061617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-bbbb0945-d120-4557-a3f8-65c2dc59e578 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:09.063640 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1019/2039] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:08 2022] DELETE /identity/v3/projects/bd69a60f677b4a1996790933f9197321 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:09.068131 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a7a8e-c7d4-4c2e-8929-be59637c5f69 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:09.068583 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a7a8e-c7d4-4c2e-8929-be59637c5f69 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:09.068937 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a7a8e-c7d4-4c2e-8929-be59637c5f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:09.091720 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-324a7a8e-c7d4-4c2e-8929-be59637c5f69 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:09.093035 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-324a7a8e-c7d4-4c2e-8929-be59637c5f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:09.093813 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e430c8-d760-4a6f-a79e-df3d24f30b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:09.094530 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e430c8-d760-4a6f-a79e-df3d24f30b26 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:09.094814 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e430c8-d760-4a6f-a79e-df3d24f30b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:09.095116 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e430c8-d760-4a6f-a79e-df3d24f30b26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:09.095810 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e430c8-d760-4a6f-a79e-df3d24f30b26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:09.097735 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e430c8-d760-4a6f-a79e-df3d24f30b26 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:09.102829 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1020/2040] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:09.110612 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ae112-16d6-482d-ac59-35e05a632fb3 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:09.111170 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ae112-16d6-482d-ac59-35e05a632fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:09.111481 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ae112-16d6-482d-ac59-35e05a632fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:09.130331 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1021/2041] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:08 2022] POST /identity/v3/projects => generated 365 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:09.135371 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e430c8-d760-4a6f-a79e-df3d24f30b26 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:09.140641 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-db0ae112-16d6-482d-ac59-35e05a632fb3 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:09.140754 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-43e430c8-d760-4a6f-a79e-df3d24f30b26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:09.142716 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-db0ae112-16d6-482d-ac59-35e05a632fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:09.166955 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:09.236503 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1021/2042] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:44:09.238315 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0ae112-16d6-482d-ac59-35e05a632fb3 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:09.249410 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0ae112-16d6-482d-ac59-35e05a632fb3 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:09.252935 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e430c8-d760-4a6f-a79e-df3d24f30b26 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:09.258564 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acbe35b-5bd2-4701-a8aa-7012aeb94070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:09.259285 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbe35b-5bd2-4701-a8aa-7012aeb94070 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:09.259629 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbe35b-5bd2-4701-a8aa-7012aeb94070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:09.260202 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbe35b-5bd2-4701-a8aa-7012aeb94070 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:09.260498 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f631f-4faf-430e-957e-a9c00c1e6c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:09.260968 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acbe35b-5bd2-4701-a8aa-7012aeb94070 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:09.261542 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f631f-4faf-430e-957e-a9c00c1e6c0b admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:09.261993 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f631f-4faf-430e-957e-a9c00c1e6c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:09.262480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f631f-4faf-430e-957e-a9c00c1e6c0b admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:09.263404 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acbe35b-5bd2-4701-a8aa-7012aeb94070 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:09.263894 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43f631f-4faf-430e-957e-a9c00c1e6c0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:44:09.264512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43f631f-4faf-430e-957e-a9c00c1e6c0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:09.266520 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43f631f-4faf-430e-957e-a9c00c1e6c0b admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:09.299080 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1022/2043] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:44:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:44:09.299418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1022/2044] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:09 2022] POST /identity/v3/users => generated 283 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:09.299499 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f631f-4faf-430e-957e-a9c00c1e6c0b admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:09.309951 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f631f-4faf-430e-957e-a9c00c1e6c0b admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:09.310405 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acbe35b-5bd2-4701-a8aa-7012aeb94070 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:09.317137 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dec5c2-992e-4cb3-a6ec-d24abe49ae89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:09.318815 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dec5c2-992e-4cb3-a6ec-d24abe49ae89 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:09.319259 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dec5c2-992e-4cb3-a6ec-d24abe49ae89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:09.319809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dec5c2-992e-4cb3-a6ec-d24abe49ae89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:09.321044 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dec5c2-992e-4cb3-a6ec-d24abe49ae89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:09.324478 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dec5c2-992e-4cb3-a6ec-d24abe49ae89 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:09.335154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acbe35b-5bd2-4701-a8aa-7012aeb94070 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:09.343619 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1023/2045] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:44:09.344875 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dec5c2-992e-4cb3-a6ec-d24abe49ae89 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:09.351548 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98c7c0-fc22-4c1f-99f1-55e63d90b370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:09.352243 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98c7c0-fc22-4c1f-99f1-55e63d90b370 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:09.352537 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98c7c0-fc22-4c1f-99f1-55e63d90b370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:09.353003 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98c7c0-fc22-4c1f-99f1-55e63d90b370 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:09.353713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a98c7c0-fc22-4c1f-99f1-55e63d90b370 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:09.356073 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a98c7c0-fc22-4c1f-99f1-55e63d90b370 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:09.358375 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dec5c2-992e-4cb3-a6ec-d24abe49ae89 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:09.368089 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eda2e5-073d-452e-87f6-dfb8dcbf9472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:09.368791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eda2e5-073d-452e-87f6-dfb8dcbf9472 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:09.369239 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eda2e5-073d-452e-87f6-dfb8dcbf9472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:09.369523 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eda2e5-073d-452e-87f6-dfb8dcbf9472 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:09.370402 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eda2e5-073d-452e-87f6-dfb8dcbf9472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:09.372093 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eda2e5-073d-452e-87f6-dfb8dcbf9472 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:09.384033 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:09.386787 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1024/2046] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:09.392498 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eda2e5-073d-452e-87f6-dfb8dcbf9472 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:09.405804 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1023/2047] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:09 2022] POST /identity/v3/projects => generated 375 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:09.407955 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eda2e5-073d-452e-87f6-dfb8dcbf9472 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:09.409762 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98c7c0-fc22-4c1f-99f1-55e63d90b370 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:09.502840 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98c7c0-fc22-4c1f-99f1-55e63d90b370 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:09.514640 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd6e7b-0d44-4767-baa7-3c2505254445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:09.515355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd6e7b-0d44-4767-baa7-3c2505254445 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:09.515641 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd6e7b-0d44-4767-baa7-3c2505254445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:09.515917 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd6e7b-0d44-4767-baa7-3c2505254445 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:09.516608 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecd6e7b-0d44-4767-baa7-3c2505254445 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:09.518536 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecd6e7b-0d44-4767-baa7-3c2505254445 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:09.521778 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c2d7e4-2d88-408e-b07b-b05e09faf865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:09.523057 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2d7e4-2d88-408e-b07b-b05e09faf865 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:09.523680 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2d7e4-2d88-408e-b07b-b05e09faf865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:09.524220 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2d7e4-2d88-408e-b07b-b05e09faf865 admin admin] PATH_INFO: `/v3/projects/b10d4715dfca43ca936c11ae82d6607c/users/c8db2c254e9141f4a7356c640e1a1a9d/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:09.540773 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1024/2048] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:09 2022] POST /identity/v3/users => generated 288 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:44:09.545022 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd6e7b-0d44-4767-baa7-3c2505254445 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:09.546935 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c2d7e4-2d88-408e-b07b-b05e09faf865 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b10d4715dfca43ca936c11ae82d6607c, user_id=c8db2c254e9141f4a7356c640e1a1a9d, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:09.553201 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c2d7e4-2d88-408e-b07b-b05e09faf865 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:09.557162 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd6e7b-0d44-4767-baa7-3c2505254445 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:09.563414 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e68839-1849-45bd-b4f4-94f7c66e3fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:09.564824 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e68839-1849-45bd-b4f4-94f7c66e3fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:09.564940 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e68839-1849-45bd-b4f4-94f7c66e3fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:09.565279 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e68839-1849-45bd-b4f4-94f7c66e3fb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:09.566284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e68839-1849-45bd-b4f4-94f7c66e3fb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:09.568203 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e68839-1849-45bd-b4f4-94f7c66e3fb9 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:09.570197 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1025/2049] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:09 2022] PUT /identity/v3/projects/b10d4715dfca43ca936c11ae82d6607c/users/c8db2c254e9141f4a7356c640e1a1a9d/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:09.581614 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8918e-b413-463c-83a5-c9cc29e8bbe1 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:09.581901 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8918e-b413-463c-83a5-c9cc29e8bbe1 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:09.582180 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8918e-b413-463c-83a5-c9cc29e8bbe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:09.586624 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1025/2050] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:44:09.587239 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e68839-1849-45bd-b4f4-94f7c66e3fb9 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:09.600977 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e68839-1849-45bd-b4f4-94f7c66e3fb9 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:09.611570 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-5de8918e-b413-463c-83a5-c9cc29e8bbe1 None None] MFA Rules not processed for user `c8db2c254e9141f4a7356c640e1a1a9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:09.612752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5de8918e-b413-463c-83a5-c9cc29e8bbe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:09.681814 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf13c8-e5e9-4c25-a204-734dfa7988e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:09.682790 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf13c8-e5e9-4c25-a204-734dfa7988e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:09.683119 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf13c8-e5e9-4c25-a204-734dfa7988e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:09.683532 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf13c8-e5e9-4c25-a204-734dfa7988e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:09.684299 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bf13c8-e5e9-4c25-a204-734dfa7988e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:09.685526 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:09.686348 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bf13c8-e5e9-4c25-a204-734dfa7988e9 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:09.693525 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1026/2051] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:09 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:44:09.699752 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1026/2052] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:44:09.707758 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8918e-b413-463c-83a5-c9cc29e8bbe1 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:09.707992 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf13c8-e5e9-4c25-a204-734dfa7988e9 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:09.717902 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf13c8-e5e9-4c25-a204-734dfa7988e9 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:09.728971 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8918e-b413-463c-83a5-c9cc29e8bbe1 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:09.731881 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713774c4-790a-4893-8aff-bf15e1275f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:09.732432 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713774c4-790a-4893-8aff-bf15e1275f14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:09.732600 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713774c4-790a-4893-8aff-bf15e1275f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:09.732960 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713774c4-790a-4893-8aff-bf15e1275f14 admin admin] PATH_INFO: `/v3/projects/ef8cf056c69f4f6da196fd17428490a6/users/96273a3f5e974eacae5b0e42c9ab0df5/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:09.736519 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fdf5a2-fbc5-4120-9946-064680a2315c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:09.737367 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fdf5a2-fbc5-4120-9946-064680a2315c admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:09.737750 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fdf5a2-fbc5-4120-9946-064680a2315c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:09.738138 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fdf5a2-fbc5-4120-9946-064680a2315c admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:09.738974 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fdf5a2-fbc5-4120-9946-064680a2315c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:09.741548 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fdf5a2-fbc5-4120-9946-064680a2315c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:09.778864 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713774c4-790a-4893-8aff-bf15e1275f14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef8cf056c69f4f6da196fd17428490a6, user_id=96273a3f5e974eacae5b0e42c9ab0df5, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:09.779551 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1027/2053] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:09 2022] POST /identity/v3/projects => generated 363 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:09.779664 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713774c4-790a-4893-8aff-bf15e1275f14 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:09.789935 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fdf5a2-fbc5-4120-9946-064680a2315c admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:09.805723 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1027/2054] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:09 2022] PUT /identity/v3/projects/ef8cf056c69f4f6da196fd17428490a6/users/96273a3f5e974eacae5b0e42c9ab0df5/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:09.818282 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90ceb8-cb72-4266-8102-b3090487f595 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:09.818650 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90ceb8-cb72-4266-8102-b3090487f595 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:09.818908 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90ceb8-cb72-4266-8102-b3090487f595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:09.884639 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-1e90ceb8-cb72-4266-8102-b3090487f595 None None] MFA Rules not processed for user `96273a3f5e974eacae5b0e42c9ab0df5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:09.888849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1e90ceb8-cb72-4266-8102-b3090487f595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:09.924088 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fdf5a2-fbc5-4120-9946-064680a2315c admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:09.947051 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:09.990176 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1028/2055] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:10.002072 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e90ceb8-cb72-4266-8102-b3090487f595 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:10.003661 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1e90ceb8-cb72-4266-8102-b3090487f595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:10.010081 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08356ec-71bd-483a-8f8d-92aa98aef546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:10.012965 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08356ec-71bd-483a-8f8d-92aa98aef546 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:10.012965 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08356ec-71bd-483a-8f8d-92aa98aef546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:10.012965 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08356ec-71bd-483a-8f8d-92aa98aef546 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:10.012965 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08356ec-71bd-483a-8f8d-92aa98aef546 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:10.019082 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08356ec-71bd-483a-8f8d-92aa98aef546 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:10.041152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1028/2056] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:09 2022] POST /identity/v3/users => generated 288 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:10.048533 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08356ec-71bd-483a-8f8d-92aa98aef546 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:10.060190 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08356ec-71bd-483a-8f8d-92aa98aef546 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:10.065118 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10440c4b-3bac-452b-a4dc-234ca6cfbb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:10.065773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10440c4b-3bac-452b-a4dc-234ca6cfbb75 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:10.066665 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10440c4b-3bac-452b-a4dc-234ca6cfbb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:10.067100 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10440c4b-3bac-452b-a4dc-234ca6cfbb75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:10.068447 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10440c4b-3bac-452b-a4dc-234ca6cfbb75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:10.071087 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10440c4b-3bac-452b-a4dc-234ca6cfbb75 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:10.094074 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e90ceb8-cb72-4266-8102-b3090487f595 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:10.098021 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1029/2057] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:10.099587 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f25d401-e7be-4c28-b56b-f6425d77cf11 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f10214b2bb4eafb3dff7adf71b9bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:10.099732 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10440c4b-3bac-452b-a4dc-234ca6cfbb75 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:10.100257 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f25d401-e7be-4c28-b56b-f6425d77cf11 service glance] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:10.100548 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f25d401-e7be-4c28-b56b-f6425d77cf11 service glance] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:10.100831 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f25d401-e7be-4c28-b56b-f6425d77cf11 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:10.111728 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10440c4b-3bac-452b-a4dc-234ca6cfbb75 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:10.113803 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f25d401-e7be-4c28-b56b-f6425d77cf11 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:10.116561 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f25d401-e7be-4c28-b56b-f6425d77cf11 service glance] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:10.119557 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f40e1-7ffa-4b26-873b-ad3f428bacce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:10.120822 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f40e1-7ffa-4b26-873b-ad3f428bacce admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:10.121436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f40e1-7ffa-4b26-873b-ad3f428bacce admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:10.121944 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f40e1-7ffa-4b26-873b-ad3f428bacce admin admin] PATH_INFO: `/v3/projects/da98037792874fab89ab34e8f9b25aef/users/99059a35e5994ef4a12dcd62447904a8/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:10.141778 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1029/2058] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:44:10.167858 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6f40e1-7ffa-4b26-873b-ad3f428bacce admin admin] RBAC: Authorizing `identity:create_grant(project_id=da98037792874fab89ab34e8f9b25aef, user_id=99059a35e5994ef4a12dcd62447904a8, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:10.171977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6f40e1-7ffa-4b26-873b-ad3f428bacce admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:10.187864 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1030/2059] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:10 2022] PUT /identity/v3/projects/da98037792874fab89ab34e8f9b25aef/users/99059a35e5994ef4a12dcd62447904a8/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:10.192937 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f25d401-e7be-4c28-b56b-f6425d77cf11 service glance] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:10.222234 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f25d401-e7be-4c28-b56b-f6425d77cf11 service glance] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:10.320353 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f597b-9fe2-4a46-9459-32b1f816cbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:10.322028 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f597b-9fe2-4a46-9459-32b1f816cbee admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:10.323300 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f597b-9fe2-4a46-9459-32b1f816cbee admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:10.325036 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f597b-9fe2-4a46-9459-32b1f816cbee admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:10.326115 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9f597b-9fe2-4a46-9459-32b1f816cbee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:10.328035 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9f597b-9fe2-4a46-9459-32b1f816cbee admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:10.352664 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1030/2060] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:10.358499 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f40e1-7ffa-4b26-873b-ad3f428bacce admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:10.389747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f40e1-7ffa-4b26-873b-ad3f428bacce admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:10.394467 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf43342-5e6f-4de3-9635-8ec8386a3086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:10.395449 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf43342-5e6f-4de3-9635-8ec8386a3086 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:10.395766 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf43342-5e6f-4de3-9635-8ec8386a3086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:10.396042 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf43342-5e6f-4de3-9635-8ec8386a3086 admin admin] PATH_INFO: `/v3/projects/da98037792874fab89ab34e8f9b25aef/users/99059a35e5994ef4a12dcd62447904a8/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:10.397997 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf43342-5e6f-4de3-9635-8ec8386a3086 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da98037792874fab89ab34e8f9b25aef, user_id=99059a35e5994ef4a12dcd62447904a8, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:10.400664 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf43342-5e6f-4de3-9635-8ec8386a3086 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:10.412241 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1031/2061] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:10 2022] PUT /identity/v3/projects/da98037792874fab89ab34e8f9b25aef/users/99059a35e5994ef4a12dcd62447904a8/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:10.422451 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62a564-e453-47b2-9bde-d6fe66cd9a6a None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:10.422942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62a564-e453-47b2-9bde-d6fe66cd9a6a None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:10.424225 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62a564-e453-47b2-9bde-d6fe66cd9a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:10.468047 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-2b62a564-e453-47b2-9bde-d6fe66cd9a6a None None] MFA Rules not processed for user `99059a35e5994ef4a12dcd62447904a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:10.469437 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-2b62a564-e453-47b2-9bde-d6fe66cd9a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:10.573977 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1031/2062] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:10 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:10.836272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf43342-5e6f-4de3-9635-8ec8386a3086 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:10.838013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-dcf43342-5e6f-4de3-9635-8ec8386a3086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:10.906426 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:10.946948 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf43342-5e6f-4de3-9635-8ec8386a3086 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:10.951185 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba892626-e9f0-4ace-8303-671f8be740d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:10.951686 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba892626-e9f0-4ace-8303-671f8be740d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:10.951971 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba892626-e9f0-4ace-8303-671f8be740d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:10.952245 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba892626-e9f0-4ace-8303-671f8be740d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:10.965538 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba892626-e9f0-4ace-8303-671f8be740d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:10.967405 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba892626-e9f0-4ace-8303-671f8be740d5 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:10.980097 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1032/2063] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:10 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:11.054852 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62a564-e453-47b2-9bde-d6fe66cd9a6a None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:11.059569 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:11.065531 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62a564-e453-47b2-9bde-d6fe66cd9a6a None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:11.069911 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e04c3a-677c-41ed-9832-00c0ed481d77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:11.070648 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e04c3a-677c-41ed-9832-00c0ed481d77 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:11.071025 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e04c3a-677c-41ed-9832-00c0ed481d77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:11.071358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e04c3a-677c-41ed-9832-00c0ed481d77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:11.134517 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e04c3a-677c-41ed-9832-00c0ed481d77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:11.136710 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e04c3a-677c-41ed-9832-00c0ed481d77 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:11.163409 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1032/2064] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:11 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:11.168890 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba892626-e9f0-4ace-8303-671f8be740d5 service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:11.171607 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ba892626-e9f0-4ace-8303-671f8be740d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:11.250488 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba892626-e9f0-4ace-8303-671f8be740d5 service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:11.255997 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8467f-d28b-4d13-aafd-6acc865d62f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:11.256876 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8467f-d28b-4d13-aafd-6acc865d62f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:11.257163 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8467f-d28b-4d13-aafd-6acc865d62f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:11.257444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8467f-d28b-4d13-aafd-6acc865d62f8 admin admin] PATH_INFO: `/v3/users/789e349e3b1c41df8494572672541885` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:11.258645 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8467f-d28b-4d13-aafd-6acc865d62f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=789e349e3b1c41df8494572672541885)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:11.260605 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8467f-d28b-4d13-aafd-6acc865d62f8 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:11.324915 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-39b8467f-d28b-4d13-aafd-6acc865d62f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '789e349e3b1c41df8494572672541885'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:11.325575 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-39b8467f-d28b-4d13-aafd-6acc865d62f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '789e349e3b1c41df8494572672541885'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:11.338248 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-39b8467f-d28b-4d13-aafd-6acc865d62f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '789e349e3b1c41df8494572672541885'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:11.338890 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-39b8467f-d28b-4d13-aafd-6acc865d62f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '789e349e3b1c41df8494572672541885'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:11.346773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-39b8467f-d28b-4d13-aafd-6acc865d62f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '789e349e3b1c41df8494572672541885'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:11.363574 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1033/2065] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:11 2022] DELETE /identity/v3/users/789e349e3b1c41df8494572672541885 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:11.368617 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06d4d8-0c5f-4e04-b72c-02031ca4aa5c None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:11.368915 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06d4d8-0c5f-4e04-b72c-02031ca4aa5c None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:11.369202 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06d4d8-0c5f-4e04-b72c-02031ca4aa5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:11.387782 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-ca06d4d8-0c5f-4e04-b72c-02031ca4aa5c None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:11.390122 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ca06d4d8-0c5f-4e04-b72c-02031ca4aa5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:11.462511 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1033/2066] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:11.472167 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8467f-d28b-4d13-aafd-6acc865d62f8 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:11.475688 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-39b8467f-d28b-4d13-aafd-6acc865d62f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:11.534844 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:11.539744 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8467f-d28b-4d13-aafd-6acc865d62f8 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:11.544315 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7888207-7dad-455e-9584-3a4162f704b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:11.544980 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7888207-7dad-455e-9584-3a4162f704b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:11.545599 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7888207-7dad-455e-9584-3a4162f704b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:11.545986 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7888207-7dad-455e-9584-3a4162f704b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:11.558177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7888207-7dad-455e-9584-3a4162f704b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:11.560454 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7888207-7dad-455e-9584-3a4162f704b5 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:11.572010 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1034/2067] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:44:11.898160 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06d4d8-0c5f-4e04-b72c-02031ca4aa5c None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:11.899717 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ca06d4d8-0c5f-4e04-b72c-02031ca4aa5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:11.914584 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06d4d8-0c5f-4e04-b72c-02031ca4aa5c None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:11.935211 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d006f19-7a29-47f0-8309-e2fda9b35c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:11.936175 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d006f19-7a29-47f0-8309-e2fda9b35c5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:11.936399 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d006f19-7a29-47f0-8309-e2fda9b35c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:11.936675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d006f19-7a29-47f0-8309-e2fda9b35c5e admin admin] PATH_INFO: `/v3/projects/77d56467f04249f6a419a0f843195965` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:11.937891 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d006f19-7a29-47f0-8309-e2fda9b35c5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=77d56467f04249f6a419a0f843195965)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:11.940573 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d006f19-7a29-47f0-8309-e2fda9b35c5e admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:12.009840 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-3d006f19-7a29-47f0-8309-e2fda9b35c5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77d56467f04249f6a419a0f843195965', 'request_id': 'req-3d006f19-7a29-47f0-8309-e2fda9b35c5e'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:12.017292 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-3d006f19-7a29-47f0-8309-e2fda9b35c5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77d56467f04249f6a419a0f843195965', 'request_id': 'req-3d006f19-7a29-47f0-8309-e2fda9b35c5e'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:12.024184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-3d006f19-7a29-47f0-8309-e2fda9b35c5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77d56467f04249f6a419a0f843195965', 'request_id': 'req-3d006f19-7a29-47f0-8309-e2fda9b35c5e'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:12.034895 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-3d006f19-7a29-47f0-8309-e2fda9b35c5e admin admin] The token cache is being invalidate because project 77d56467f04249f6a419a0f843195965 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:12.035215 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-3d006f19-7a29-47f0-8309-e2fda9b35c5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:12.035852 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-3d006f19-7a29-47f0-8309-e2fda9b35c5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:12.037975 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1034/2068] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:11 2022] DELETE /identity/v3/projects/77d56467f04249f6a419a0f843195965 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:12.043170 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b8182-75b3-4121-8f5f-20067e79bb5e None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:12.043619 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b8182-75b3-4121-8f5f-20067e79bb5e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:12.043709 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b8182-75b3-4121-8f5f-20067e79bb5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:12.061169 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-0a1b8182-75b3-4121-8f5f-20067e79bb5e None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:12.062364 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-0a1b8182-75b3-4121-8f5f-20067e79bb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:12.119148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1035/2069] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:12.127881 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4659eaa-7cc2-4749-a996-9d7687cf443b None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:12.128464 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4659eaa-7cc2-4749-a996-9d7687cf443b None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:12.128575 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4659eaa-7cc2-4749-a996-9d7687cf443b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:12.147688 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-e4659eaa-7cc2-4749-a996-9d7687cf443b None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:12.148875 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e4659eaa-7cc2-4749-a996-9d7687cf443b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:12.156955 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1035/2070] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:12.161731 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b8182-75b3-4121-8f5f-20067e79bb5e None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:12.171384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b8182-75b3-4121-8f5f-20067e79bb5e None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:12.176089 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0193d13-6a5f-4778-aa63-57fa21a74fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:12.176770 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0193d13-6a5f-4778-aa63-57fa21a74fc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:12.177060 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0193d13-6a5f-4778-aa63-57fa21a74fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:12.177337 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0193d13-6a5f-4778-aa63-57fa21a74fc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:12.178068 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0193d13-6a5f-4778-aa63-57fa21a74fc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:44:12.178377 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0193d13-6a5f-4778-aa63-57fa21a74fc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:12.180020 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0193d13-6a5f-4778-aa63-57fa21a74fc5 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:12.191430 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1036/2071] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:44:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:44:12.196464 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4659eaa-7cc2-4749-a996-9d7687cf443b None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:12.205974 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4659eaa-7cc2-4749-a996-9d7687cf443b None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:12.211501 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7da5213-25a0-4c8a-824b-71cf1363b072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:12.212157 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da5213-25a0-4c8a-824b-71cf1363b072 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:12.212438 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da5213-25a0-4c8a-824b-71cf1363b072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:12.212714 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da5213-25a0-4c8a-824b-71cf1363b072 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:12.213387 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7da5213-25a0-4c8a-824b-71cf1363b072 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:12.215319 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7da5213-25a0-4c8a-824b-71cf1363b072 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:12.230006 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:12.244656 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1036/2072] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:12 2022] POST /identity/v3/projects => generated 399 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:12.248744 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0193d13-6a5f-4778-aa63-57fa21a74fc5 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:12.259365 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0193d13-6a5f-4778-aa63-57fa21a74fc5 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:12.317122 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead51da-7b7a-4cf8-9e3a-bd011634e784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:12.318361 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead51da-7b7a-4cf8-9e3a-bd011634e784 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:12.318630 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead51da-7b7a-4cf8-9e3a-bd011634e784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:12.318908 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead51da-7b7a-4cf8-9e3a-bd011634e784 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:12.320002 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead51da-7b7a-4cf8-9e3a-bd011634e784 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:12.321575 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead51da-7b7a-4cf8-9e3a-bd011634e784 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:12.348399 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1037/2073] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:12 2022] POST /identity/v3/users => generated 300 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:12.353284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7da5213-25a0-4c8a-824b-71cf1363b072 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:12.363459 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7da5213-25a0-4c8a-824b-71cf1363b072 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:12.368805 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c75b5c-401e-4cf6-932c-51341d79187f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:12.369457 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c75b5c-401e-4cf6-932c-51341d79187f admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:12.369741 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c75b5c-401e-4cf6-932c-51341d79187f admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:12.370012 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c75b5c-401e-4cf6-932c-51341d79187f admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:12.370729 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c75b5c-401e-4cf6-932c-51341d79187f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:12.372375 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c75b5c-401e-4cf6-932c-51341d79187f admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:12.383281 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1037/2074] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:12.387257 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead51da-7b7a-4cf8-9e3a-bd011634e784 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:12.397327 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead51da-7b7a-4cf8-9e3a-bd011634e784 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:12.402490 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500f645-ba92-436e-a9ba-a5aa557e3f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:12.403189 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500f645-ba92-436e-a9ba-a5aa557e3f5c admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:12.403474 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500f645-ba92-436e-a9ba-a5aa557e3f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:12.403752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500f645-ba92-436e-a9ba-a5aa557e3f5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:12.404481 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500f645-ba92-436e-a9ba-a5aa557e3f5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:12.406085 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500f645-ba92-436e-a9ba-a5aa557e3f5c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:12.416483 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1038/2075] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:12.420654 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c75b5c-401e-4cf6-932c-51341d79187f admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:12.431285 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c75b5c-401e-4cf6-932c-51341d79187f admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:12.435906 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70bd039-cfc5-4d6f-9657-65f98d58e76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:12.436810 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70bd039-cfc5-4d6f-9657-65f98d58e76d admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:12.437089 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70bd039-cfc5-4d6f-9657-65f98d58e76d admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:12.437360 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70bd039-cfc5-4d6f-9657-65f98d58e76d admin admin] PATH_INFO: `/v3/projects/1aebf7ec22624bfc8223ae89b9bc7232/users/a07c65af8c9f43c2aab84321fed56a50/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:12.452391 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70bd039-cfc5-4d6f-9657-65f98d58e76d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aebf7ec22624bfc8223ae89b9bc7232, user_id=a07c65af8c9f43c2aab84321fed56a50, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:12.455075 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70bd039-cfc5-4d6f-9657-65f98d58e76d admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:12.468509 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1038/2076] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:12 2022] PUT /identity/v3/projects/1aebf7ec22624bfc8223ae89b9bc7232/users/a07c65af8c9f43c2aab84321fed56a50/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:12.479257 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44b42b-b732-4069-bafe-ee07df026bd7 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:12.479624 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44b42b-b732-4069-bafe-ee07df026bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:12.479971 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44b42b-b732-4069-bafe-ee07df026bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:12.506428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-ed44b42b-b732-4069-bafe-ee07df026bd7 None None] MFA Rules not processed for user `a07c65af8c9f43c2aab84321fed56a50`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:12.507669 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ed44b42b-b732-4069-bafe-ee07df026bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:12.550832 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:12.574581 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1039/2077] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:12 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:12.586252 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70bd039-cfc5-4d6f-9657-65f98d58e76d admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:12.587929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c70bd039-cfc5-4d6f-9657-65f98d58e76d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:12.647035 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70bd039-cfc5-4d6f-9657-65f98d58e76d admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:12.656386 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f8675-4058-415b-864d-ce5ce2d46a37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:12.657897 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f8675-4058-415b-864d-ce5ce2d46a37 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:12.658194 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f8675-4058-415b-864d-ce5ce2d46a37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:12.658477 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f8675-4058-415b-864d-ce5ce2d46a37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:12.672738 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4f8675-4058-415b-864d-ce5ce2d46a37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:12.674181 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4f8675-4058-415b-864d-ce5ce2d46a37 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:12.684452 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1039/2078] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:12 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:22.571718 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=6 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:22.680674 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=43,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:35.381914 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44b42b-b732-4069-bafe-ee07df026bd7 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:35.463358 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44b42b-b732-4069-bafe-ee07df026bd7 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:35.468865 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16df836a-1e34-47d2-8c8c-192f5592d06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:35.470049 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df836a-1e34-47d2-8c8c-192f5592d06b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:35.470335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df836a-1e34-47d2-8c8c-192f5592d06b admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:35.470618 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df836a-1e34-47d2-8c8c-192f5592d06b admin admin] PATH_INFO: `/v3/users/a07c65af8c9f43c2aab84321fed56a50` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:35.472830 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16df836a-1e34-47d2-8c8c-192f5592d06b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a07c65af8c9f43c2aab84321fed56a50)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:35.475421 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16df836a-1e34-47d2-8c8c-192f5592d06b admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:35.527662 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-16df836a-1e34-47d2-8c8c-192f5592d06b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a07c65af8c9f43c2aab84321fed56a50'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:35.528585 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-16df836a-1e34-47d2-8c8c-192f5592d06b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a07c65af8c9f43c2aab84321fed56a50'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:35.542137 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-16df836a-1e34-47d2-8c8c-192f5592d06b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a07c65af8c9f43c2aab84321fed56a50'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:35.542830 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-16df836a-1e34-47d2-8c8c-192f5592d06b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a07c65af8c9f43c2aab84321fed56a50'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:35.556011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-16df836a-1e34-47d2-8c8c-192f5592d06b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a07c65af8c9f43c2aab84321fed56a50'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:35.571511 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1040/2079] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:35 2022] DELETE /identity/v3/users/a07c65af8c9f43c2aab84321fed56a50 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:35.576253 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1d9d9-2584-4c42-91a8-554d4fd15135 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:35.576759 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1d9d9-2584-4c42-91a8-554d4fd15135 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:35.576922 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1d9d9-2584-4c42-91a8-554d4fd15135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:35.597025 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-d8e1d9d9-2584-4c42-91a8-554d4fd15135 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:35.598043 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d8e1d9d9-2584-4c42-91a8-554d4fd15135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:35.663161 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1040/2080] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:35.676556 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16df836a-1e34-47d2-8c8c-192f5592d06b admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:35.678677 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-16df836a-1e34-47d2-8c8c-192f5592d06b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:35.775308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16df836a-1e34-47d2-8c8c-192f5592d06b admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:35.780502 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eefb45-fd24-4b36-b177-487017d69bd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:35.781167 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eefb45-fd24-4b36-b177-487017d69bd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:35.781452 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eefb45-fd24-4b36-b177-487017d69bd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:35.781747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eefb45-fd24-4b36-b177-487017d69bd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:35.791910 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eefb45-fd24-4b36-b177-487017d69bd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:35.793467 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eefb45-fd24-4b36-b177-487017d69bd7 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:35.804692 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1041/2081] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:36.169646 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1d9d9-2584-4c42-91a8-554d4fd15135 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:36.171673 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d8e1d9d9-2584-4c42-91a8-554d4fd15135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:36.188188 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1d9d9-2584-4c42-91a8-554d4fd15135 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:36.196394 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628377aa-642b-48fa-ba65-6cce6d9b0791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:36.197557 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628377aa-642b-48fa-ba65-6cce6d9b0791 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:36.199093 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628377aa-642b-48fa-ba65-6cce6d9b0791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:36.199476 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628377aa-642b-48fa-ba65-6cce6d9b0791 admin admin] PATH_INFO: `/v3/projects/1aebf7ec22624bfc8223ae89b9bc7232` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:36.200895 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628377aa-642b-48fa-ba65-6cce6d9b0791 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1aebf7ec22624bfc8223ae89b9bc7232)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:36.205328 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628377aa-642b-48fa-ba65-6cce6d9b0791 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:36.267142 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-628377aa-642b-48fa-ba65-6cce6d9b0791 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1aebf7ec22624bfc8223ae89b9bc7232', 'request_id': 'req-628377aa-642b-48fa-ba65-6cce6d9b0791'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:36.277837 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-628377aa-642b-48fa-ba65-6cce6d9b0791 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1aebf7ec22624bfc8223ae89b9bc7232', 'request_id': 'req-628377aa-642b-48fa-ba65-6cce6d9b0791'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:36.288074 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-628377aa-642b-48fa-ba65-6cce6d9b0791 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1aebf7ec22624bfc8223ae89b9bc7232', 'request_id': 'req-628377aa-642b-48fa-ba65-6cce6d9b0791'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:36.310243 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-628377aa-642b-48fa-ba65-6cce6d9b0791 admin admin] The token cache is being invalidate because project 1aebf7ec22624bfc8223ae89b9bc7232 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:36.310584 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-628377aa-642b-48fa-ba65-6cce6d9b0791 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:36.311570 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-628377aa-642b-48fa-ba65-6cce6d9b0791 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:36.313783 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1041/2082] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:36 2022] DELETE /identity/v3/projects/1aebf7ec22624bfc8223ae89b9bc7232 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:36.935910 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eefb45-fd24-4b36-b177-487017d69bd7 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:36.938215 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a1eefb45-fd24-4b36-b177-487017d69bd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:36.955837 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eefb45-fd24-4b36-b177-487017d69bd7 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:36.977307 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:37.031558 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50da7b-bb08-4477-b7a2-50d07bac4599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:37.032445 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50da7b-bb08-4477-b7a2-50d07bac4599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:37.033492 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50da7b-bb08-4477-b7a2-50d07bac4599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:37.033977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50da7b-bb08-4477-b7a2-50d07bac4599 admin admin] PATH_INFO: `/v3/users/e89c9a7f88b04fdbac6a3927241dd4ce` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:37.036078 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d50da7b-bb08-4477-b7a2-50d07bac4599 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e89c9a7f88b04fdbac6a3927241dd4ce)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:37.037610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d50da7b-bb08-4477-b7a2-50d07bac4599 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:37.109741 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4d50da7b-bb08-4477-b7a2-50d07bac4599 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e89c9a7f88b04fdbac6a3927241dd4ce'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:37.110350 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4d50da7b-bb08-4477-b7a2-50d07bac4599 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e89c9a7f88b04fdbac6a3927241dd4ce'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:37.123443 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4d50da7b-bb08-4477-b7a2-50d07bac4599 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e89c9a7f88b04fdbac6a3927241dd4ce'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:37.124176 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4d50da7b-bb08-4477-b7a2-50d07bac4599 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e89c9a7f88b04fdbac6a3927241dd4ce'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:37.134078 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4d50da7b-bb08-4477-b7a2-50d07bac4599 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e89c9a7f88b04fdbac6a3927241dd4ce'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:37.163062 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055cadc7-4bc7-4a35-b4b1-6611df8819c5 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:37.163981 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055cadc7-4bc7-4a35-b4b1-6611df8819c5 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:37.164498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055cadc7-4bc7-4a35-b4b1-6611df8819c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:37.166462 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1042/2083] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:36 2022] DELETE /identity/v3/users/e89c9a7f88b04fdbac6a3927241dd4ce => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:37.187079 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-055cadc7-4bc7-4a35-b4b1-6611df8819c5 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:37.189184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-055cadc7-4bc7-4a35-b4b1-6611df8819c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:37.276342 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1042/2084] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:37.285665 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50da7b-bb08-4477-b7a2-50d07bac4599 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:37.287812 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-4d50da7b-bb08-4477-b7a2-50d07bac4599 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:37.403691 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50da7b-bb08-4477-b7a2-50d07bac4599 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:37.409099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defeb775-6068-4034-9734-4dfbbc199177 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:37.410154 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defeb775-6068-4034-9734-4dfbbc199177 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:37.410813 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defeb775-6068-4034-9734-4dfbbc199177 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:37.411431 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defeb775-6068-4034-9734-4dfbbc199177 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:37.430631 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defeb775-6068-4034-9734-4dfbbc199177 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:37.434636 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defeb775-6068-4034-9734-4dfbbc199177 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:37.454937 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1043/2085] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:38.055050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055cadc7-4bc7-4a35-b4b1-6611df8819c5 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:38.056837 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-055cadc7-4bc7-4a35-b4b1-6611df8819c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:38.072782 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055cadc7-4bc7-4a35-b4b1-6611df8819c5 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:38.079105 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f8e52-a07a-4df2-a0f8-0c35c761cfc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:38.080067 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f8e52-a07a-4df2-a0f8-0c35c761cfc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:38.080376 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f8e52-a07a-4df2-a0f8-0c35c761cfc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:38.080659 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f8e52-a07a-4df2-a0f8-0c35c761cfc4 admin admin] PATH_INFO: `/v3/projects/73bd4e75069d41d3bb9ebfb932814f08` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:38.081955 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9f8e52-a07a-4df2-a0f8-0c35c761cfc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73bd4e75069d41d3bb9ebfb932814f08)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:38.085034 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9f8e52-a07a-4df2-a0f8-0c35c761cfc4 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:38.122976 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:38.162867 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0d9f8e52-a07a-4df2-a0f8-0c35c761cfc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73bd4e75069d41d3bb9ebfb932814f08', 'request_id': 'req-0d9f8e52-a07a-4df2-a0f8-0c35c761cfc4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:38.175217 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0d9f8e52-a07a-4df2-a0f8-0c35c761cfc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73bd4e75069d41d3bb9ebfb932814f08', 'request_id': 'req-0d9f8e52-a07a-4df2-a0f8-0c35c761cfc4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:38.183572 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0d9f8e52-a07a-4df2-a0f8-0c35c761cfc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73bd4e75069d41d3bb9ebfb932814f08', 'request_id': 'req-0d9f8e52-a07a-4df2-a0f8-0c35c761cfc4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:38.195878 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0d9f8e52-a07a-4df2-a0f8-0c35c761cfc4 admin admin] The token cache is being invalidate because project 73bd4e75069d41d3bb9ebfb932814f08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:38.196263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0d9f8e52-a07a-4df2-a0f8-0c35c761cfc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:38.196998 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0d9f8e52-a07a-4df2-a0f8-0c35c761cfc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:38.198727 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1043/2086] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:38 2022] DELETE /identity/v3/projects/73bd4e75069d41d3bb9ebfb932814f08 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:38.203733 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ce3d5-84f8-4eb1-a1d1-7f9591def97f None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:38.204271 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ce3d5-84f8-4eb1-a1d1-7f9591def97f None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:38.204271 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ce3d5-84f8-4eb1-a1d1-7f9591def97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:38.225248 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-970ce3d5-84f8-4eb1-a1d1-7f9591def97f None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:38.226764 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-970ce3d5-84f8-4eb1-a1d1-7f9591def97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:38.276089 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:38.305027 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1044/2087] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:38.312495 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5664edc-0f23-4006-9b88-0d7a69fcbbd0 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:38.312784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5664edc-0f23-4006-9b88-0d7a69fcbbd0 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:38.313064 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5664edc-0f23-4006-9b88-0d7a69fcbbd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:38.332726 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-a5664edc-0f23-4006-9b88-0d7a69fcbbd0 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:38.334488 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-a5664edc-0f23-4006-9b88-0d7a69fcbbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:38.343689 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1044/2088] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:38.347824 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ce3d5-84f8-4eb1-a1d1-7f9591def97f None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:38.359262 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ce3d5-84f8-4eb1-a1d1-7f9591def97f None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:38.365586 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cddd643-2563-47cc-a39b-4eb170feb5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:38.365586 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cddd643-2563-47cc-a39b-4eb170feb5bb admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:38.366000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cddd643-2563-47cc-a39b-4eb170feb5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:38.366548 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cddd643-2563-47cc-a39b-4eb170feb5bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:38.367084 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cddd643-2563-47cc-a39b-4eb170feb5bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:44:38.367496 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cddd643-2563-47cc-a39b-4eb170feb5bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:38.369186 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cddd643-2563-47cc-a39b-4eb170feb5bb admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:38.383940 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1045/2089] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:44:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:44:38.387998 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5664edc-0f23-4006-9b88-0d7a69fcbbd0 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:38.398834 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5664edc-0f23-4006-9b88-0d7a69fcbbd0 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:38.403841 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7bfa5c-1512-48e3-ae64-6540ca16d151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:38.404511 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bfa5c-1512-48e3-ae64-6540ca16d151 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:38.404802 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bfa5c-1512-48e3-ae64-6540ca16d151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:38.405077 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bfa5c-1512-48e3-ae64-6540ca16d151 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:38.405770 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7bfa5c-1512-48e3-ae64-6540ca16d151 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:38.407749 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7bfa5c-1512-48e3-ae64-6540ca16d151 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:38.437626 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1045/2090] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:38 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:38.441413 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cddd643-2563-47cc-a39b-4eb170feb5bb admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:38.454216 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cddd643-2563-47cc-a39b-4eb170feb5bb admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:38.520495 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed36134-f197-4b1e-9794-386c71866561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:38.521867 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed36134-f197-4b1e-9794-386c71866561 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:38.522221 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed36134-f197-4b1e-9794-386c71866561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:38.522492 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed36134-f197-4b1e-9794-386c71866561 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:38.523233 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed36134-f197-4b1e-9794-386c71866561 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:38.525177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed36134-f197-4b1e-9794-386c71866561 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:38.558434 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1046/2091] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:38 2022] POST /identity/v3/users => generated 286 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:38.563833 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7bfa5c-1512-48e3-ae64-6540ca16d151 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:38.574854 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7bfa5c-1512-48e3-ae64-6540ca16d151 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:38.579697 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8aef98-7871-44e3-b0f5-f502d8c7a0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:38.580355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8aef98-7871-44e3-b0f5-f502d8c7a0e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:38.580635 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8aef98-7871-44e3-b0f5-f502d8c7a0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:38.580910 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8aef98-7871-44e3-b0f5-f502d8c7a0e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:38.581637 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8aef98-7871-44e3-b0f5-f502d8c7a0e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:38.583264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8aef98-7871-44e3-b0f5-f502d8c7a0e4 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:38.594609 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1046/2092] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:38.603013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed36134-f197-4b1e-9794-386c71866561 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:38.614173 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed36134-f197-4b1e-9794-386c71866561 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:38.618984 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486157b9-b279-4d62-8ecc-c6dfbb66d144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:38.619672 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486157b9-b279-4d62-8ecc-c6dfbb66d144 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:38.619954 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486157b9-b279-4d62-8ecc-c6dfbb66d144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:38.620225 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486157b9-b279-4d62-8ecc-c6dfbb66d144 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:38.620948 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486157b9-b279-4d62-8ecc-c6dfbb66d144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:38.622572 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486157b9-b279-4d62-8ecc-c6dfbb66d144 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:38.640094 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1047/2093] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:44:38.641727 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8aef98-7871-44e3-b0f5-f502d8c7a0e4 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:38.658982 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8aef98-7871-44e3-b0f5-f502d8c7a0e4 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:38.666899 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed29900-7836-4e7f-b455-a4a8a355e764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:38.666899 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed29900-7836-4e7f-b455-a4a8a355e764 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:38.666899 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed29900-7836-4e7f-b455-a4a8a355e764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:38.666899 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed29900-7836-4e7f-b455-a4a8a355e764 admin admin] PATH_INFO: `/v3/projects/8207bf7c9f5a46c883aee6dbffc11d44/users/fd5b8787942d4d5b8e23fcedc59a06a3/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:38.683204 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed29900-7836-4e7f-b455-a4a8a355e764 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8207bf7c9f5a46c883aee6dbffc11d44, user_id=fd5b8787942d4d5b8e23fcedc59a06a3, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:38.685951 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed29900-7836-4e7f-b455-a4a8a355e764 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:38.699972 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1047/2094] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:38 2022] PUT /identity/v3/projects/8207bf7c9f5a46c883aee6dbffc11d44/users/fd5b8787942d4d5b8e23fcedc59a06a3/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:38.710935 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7ed7c-6ad3-4a04-ac6c-9f1602c24a66 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:38.711263 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7ed7c-6ad3-4a04-ac6c-9f1602c24a66 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:38.711546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7ed7c-6ad3-4a04-ac6c-9f1602c24a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:38.738268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-f4e7ed7c-6ad3-4a04-ac6c-9f1602c24a66 None None] MFA Rules not processed for user `fd5b8787942d4d5b8e23fcedc59a06a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:38.739536 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f4e7ed7c-6ad3-4a04-ac6c-9f1602c24a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:38.807616 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1048/2095] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:38.826815 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed29900-7836-4e7f-b455-a4a8a355e764 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:38.828987 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-7ed29900-7836-4e7f-b455-a4a8a355e764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:38.901614 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed29900-7836-4e7f-b455-a4a8a355e764 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:38.905970 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a182f119-f39d-47d5-b0c8-fef44cc9a26c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:38.906744 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a182f119-f39d-47d5-b0c8-fef44cc9a26c service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:38.907248 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a182f119-f39d-47d5-b0c8-fef44cc9a26c service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:38.907582 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a182f119-f39d-47d5-b0c8-fef44cc9a26c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:38.926058 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a182f119-f39d-47d5-b0c8-fef44cc9a26c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:38.928287 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a182f119-f39d-47d5-b0c8-fef44cc9a26c service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:38.938888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1048/2096] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:38.963346 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e7ed7c-6ad3-4a04-ac6c-9f1602c24a66 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:39.011611 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:39.016941 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a182f119-f39d-47d5-b0c8-fef44cc9a26c service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:39.018547 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-a182f119-f39d-47d5-b0c8-fef44cc9a26c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:39.040122 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e7ed7c-6ad3-4a04-ac6c-9f1602c24a66 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:39.047042 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7700f4e-0b55-4f1b-beeb-7c8aa004b49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:39.052805 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7700f4e-0b55-4f1b-beeb-7c8aa004b49c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:39.052805 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7700f4e-0b55-4f1b-beeb-7c8aa004b49c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:39.052805 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7700f4e-0b55-4f1b-beeb-7c8aa004b49c admin admin] PATH_INFO: `/v3/users/fd5b8787942d4d5b8e23fcedc59a06a3` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:39.052805 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7700f4e-0b55-4f1b-beeb-7c8aa004b49c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd5b8787942d4d5b8e23fcedc59a06a3)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:39.052805 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7700f4e-0b55-4f1b-beeb-7c8aa004b49c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:39.092663 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:39.111487 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b7700f4e-0b55-4f1b-beeb-7c8aa004b49c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd5b8787942d4d5b8e23fcedc59a06a3'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:39.112235 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b7700f4e-0b55-4f1b-beeb-7c8aa004b49c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd5b8787942d4d5b8e23fcedc59a06a3'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:39.119569 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a182f119-f39d-47d5-b0c8-fef44cc9a26c service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:39.124500 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c338962-2219-4aea-bb98-63bad8f27475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:39.125564 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c338962-2219-4aea-bb98-63bad8f27475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:39.125849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c338962-2219-4aea-bb98-63bad8f27475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:39.126124 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c338962-2219-4aea-bb98-63bad8f27475 admin admin] PATH_INFO: `/v3/users/c8db2c254e9141f4a7356c640e1a1a9d` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:39.126739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b7700f4e-0b55-4f1b-beeb-7c8aa004b49c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd5b8787942d4d5b8e23fcedc59a06a3'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:39.127479 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b7700f4e-0b55-4f1b-beeb-7c8aa004b49c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd5b8787942d4d5b8e23fcedc59a06a3'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:39.127602 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c338962-2219-4aea-bb98-63bad8f27475 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8db2c254e9141f4a7356c640e1a1a9d)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:39.129489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c338962-2219-4aea-bb98-63bad8f27475 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:39.136982 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b7700f4e-0b55-4f1b-beeb-7c8aa004b49c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd5b8787942d4d5b8e23fcedc59a06a3'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:39.171942 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1049/2097] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:38 2022] DELETE /identity/v3/users/fd5b8787942d4d5b8e23fcedc59a06a3 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:39.176899 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93c8e1-d78b-4c0a-8fc7-844e2bb43e05 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:39.177698 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93c8e1-d78b-4c0a-8fc7-844e2bb43e05 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:39.180699 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93c8e1-d78b-4c0a-8fc7-844e2bb43e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:39.211681 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-4a93c8e1-d78b-4c0a-8fc7-844e2bb43e05 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:39.213219 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-7c338962-2219-4aea-bb98-63bad8f27475 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8db2c254e9141f4a7356c640e1a1a9d'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:39.213608 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-4a93c8e1-d78b-4c0a-8fc7-844e2bb43e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:39.213918 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-7c338962-2219-4aea-bb98-63bad8f27475 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8db2c254e9141f4a7356c640e1a1a9d'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:39.228619 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-7c338962-2219-4aea-bb98-63bad8f27475 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8db2c254e9141f4a7356c640e1a1a9d'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:39.238240 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-7c338962-2219-4aea-bb98-63bad8f27475 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8db2c254e9141f4a7356c640e1a1a9d'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:39.246953 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-7c338962-2219-4aea-bb98-63bad8f27475 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8db2c254e9141f4a7356c640e1a1a9d'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:39.258668 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1049/2098] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:39 2022] DELETE /identity/v3/users/c8db2c254e9141f4a7356c640e1a1a9d => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:39.261416 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367e1b5-46b7-4beb-a5ba-f10fd5dab845 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:39.261734 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367e1b5-46b7-4beb-a5ba-f10fd5dab845 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:39.262019 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367e1b5-46b7-4beb-a5ba-f10fd5dab845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:39.282686 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-a367e1b5-46b7-4beb-a5ba-f10fd5dab845 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:39.284140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-a367e1b5-46b7-4beb-a5ba-f10fd5dab845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:39.302551 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1050/2099] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:44:39.316629 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93c8e1-d78b-4c0a-8fc7-844e2bb43e05 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:39.319040 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-4a93c8e1-d78b-4c0a-8fc7-844e2bb43e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:39.374742 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1050/2100] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:39.386375 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a367e1b5-46b7-4beb-a5ba-f10fd5dab845 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:39.387905 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-a367e1b5-46b7-4beb-a5ba-f10fd5dab845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:39.396867 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93c8e1-d78b-4c0a-8fc7-844e2bb43e05 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:39.409756 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:39.453107 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7b6b8-6b35-4d3b-85df-1c0aa911242a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:39.453804 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7b6b8-6b35-4d3b-85df-1c0aa911242a service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:39.454156 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7b6b8-6b35-4d3b-85df-1c0aa911242a service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:39.454641 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7b6b8-6b35-4d3b-85df-1c0aa911242a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:39.456774 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a367e1b5-46b7-4beb-a5ba-f10fd5dab845 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:39.465091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60baab25-0aaa-4d72-b3e3-99610fa0de44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:39.465763 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60baab25-0aaa-4d72-b3e3-99610fa0de44 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:39.466046 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60baab25-0aaa-4d72-b3e3-99610fa0de44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:39.466383 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60baab25-0aaa-4d72-b3e3-99610fa0de44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:39.468076 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7b6b8-6b35-4d3b-85df-1c0aa911242a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:39.469692 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7b6b8-6b35-4d3b-85df-1c0aa911242a service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:39.476474 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60baab25-0aaa-4d72-b3e3-99610fa0de44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:39.477912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60baab25-0aaa-4d72-b3e3-99610fa0de44 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:39.480116 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1051/2101] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:39.488469 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1051/2102] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:39.790004 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7b6b8-6b35-4d3b-85df-1c0aa911242a service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:39.792771 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-54e7b6b8-6b35-4d3b-85df-1c0aa911242a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:39.803409 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60baab25-0aaa-4d72-b3e3-99610fa0de44 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:39.805164 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-60baab25-0aaa-4d72-b3e3-99610fa0de44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:39.810670 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7b6b8-6b35-4d3b-85df-1c0aa911242a service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:39.816149 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b0ff1-0a13-49f8-a0e0-a59553a9d87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:39.817115 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b0ff1-0a13-49f8-a0e0-a59553a9d87b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:39.817490 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b0ff1-0a13-49f8-a0e0-a59553a9d87b admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:39.818426 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b0ff1-0a13-49f8-a0e0-a59553a9d87b admin admin] PATH_INFO: `/v3/projects/8207bf7c9f5a46c883aee6dbffc11d44` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:39.819768 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32b0ff1-0a13-49f8-a0e0-a59553a9d87b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8207bf7c9f5a46c883aee6dbffc11d44)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:39.821599 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60baab25-0aaa-4d72-b3e3-99610fa0de44 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:39.822027 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32b0ff1-0a13-49f8-a0e0-a59553a9d87b admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:39.830804 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f24020-6373-44f0-b431-f69252ac0546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:39.831890 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f24020-6373-44f0-b431-f69252ac0546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:39.832166 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f24020-6373-44f0-b431-f69252ac0546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:39.832458 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f24020-6373-44f0-b431-f69252ac0546 admin admin] PATH_INFO: `/v3/projects/b10d4715dfca43ca936c11ae82d6607c` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:39.834367 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f24020-6373-44f0-b431-f69252ac0546 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b10d4715dfca43ca936c11ae82d6607c)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:39.836219 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f24020-6373-44f0-b431-f69252ac0546 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:39.839907 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:39.879534 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d32b0ff1-0a13-49f8-a0e0-a59553a9d87b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8207bf7c9f5a46c883aee6dbffc11d44', 'request_id': 'req-d32b0ff1-0a13-49f8-a0e0-a59553a9d87b'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:39.888580 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d32b0ff1-0a13-49f8-a0e0-a59553a9d87b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8207bf7c9f5a46c883aee6dbffc11d44', 'request_id': 'req-d32b0ff1-0a13-49f8-a0e0-a59553a9d87b'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:39.895831 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d32b0ff1-0a13-49f8-a0e0-a59553a9d87b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8207bf7c9f5a46c883aee6dbffc11d44', 'request_id': 'req-d32b0ff1-0a13-49f8-a0e0-a59553a9d87b'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:39.903975 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-80f24020-6373-44f0-b431-f69252ac0546 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b10d4715dfca43ca936c11ae82d6607c', 'request_id': 'req-80f24020-6373-44f0-b431-f69252ac0546'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:39.910842 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d32b0ff1-0a13-49f8-a0e0-a59553a9d87b admin admin] The token cache is being invalidate because project 8207bf7c9f5a46c883aee6dbffc11d44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:39.911341 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d32b0ff1-0a13-49f8-a0e0-a59553a9d87b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:39.912547 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-80f24020-6373-44f0-b431-f69252ac0546 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b10d4715dfca43ca936c11ae82d6607c', 'request_id': 'req-80f24020-6373-44f0-b431-f69252ac0546'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:39.913019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d32b0ff1-0a13-49f8-a0e0-a59553a9d87b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:39.916032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1052/2103] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:39 2022] DELETE /identity/v3/projects/8207bf7c9f5a46c883aee6dbffc11d44 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:39.920934 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-80f24020-6373-44f0-b431-f69252ac0546 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b10d4715dfca43ca936c11ae82d6607c', 'request_id': 'req-80f24020-6373-44f0-b431-f69252ac0546'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:39.921450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf5e7e-7198-4b80-b78b-1c4d4f0a29a3 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:39.921858 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf5e7e-7198-4b80-b78b-1c4d4f0a29a3 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:39.922255 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf5e7e-7198-4b80-b78b-1c4d4f0a29a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:39.938026 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-80f24020-6373-44f0-b431-f69252ac0546 admin admin] The token cache is being invalidate because project b10d4715dfca43ca936c11ae82d6607c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:39.938321 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-80f24020-6373-44f0-b431-f69252ac0546 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:39.939013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-80f24020-6373-44f0-b431-f69252ac0546 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:39.940651 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1052/2104] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:39 2022] DELETE /identity/v3/projects/b10d4715dfca43ca936c11ae82d6607c => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:39.945621 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f24020-6373-44f0-b431-f69252ac0546 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:39.947138 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-80f24020-6373-44f0-b431-f69252ac0546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:39.947414 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-33cf5e7e-7198-4b80-b78b-1c4d4f0a29a3 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:39.948685 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-33cf5e7e-7198-4b80-b78b-1c4d4f0a29a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:40.016943 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1053/2105] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:44:40.023502 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f24020-6373-44f0-b431-f69252ac0546 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:40.026606 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec5846-8486-4fa4-80ba-6b78a74b171c None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:40.026904 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec5846-8486-4fa4-80ba-6b78a74b171c None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:40.027260 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec5846-8486-4fa4-80ba-6b78a74b171c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:40.029642 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea4850c-2f78-4fda-b46f-757debcd326f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:40.030734 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4850c-2f78-4fda-b46f-757debcd326f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:40.031229 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4850c-2f78-4fda-b46f-757debcd326f admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:40.031837 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4850c-2f78-4fda-b46f-757debcd326f admin admin] PATH_INFO: `/v3/users/99059a35e5994ef4a12dcd62447904a8` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:40.033293 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea4850c-2f78-4fda-b46f-757debcd326f admin admin] RBAC: Authorizing `identity:delete_user(user_id=99059a35e5994ef4a12dcd62447904a8)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:40.035466 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea4850c-2f78-4fda-b46f-757debcd326f admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:40.048102 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-57ec5846-8486-4fa4-80ba-6b78a74b171c None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:40.049743 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-57ec5846-8486-4fa4-80ba-6b78a74b171c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:40.058802 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1054/2106] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:44:40.062628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec5846-8486-4fa4-80ba-6b78a74b171c None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:40.073581 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec5846-8486-4fa4-80ba-6b78a74b171c None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:40.079333 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e1ea3-e74d-490c-934d-4165ae50624f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:40.080036 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e1ea3-e74d-490c-934d-4165ae50624f admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:40.080323 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e1ea3-e74d-490c-934d-4165ae50624f admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:40.080601 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e1ea3-e74d-490c-934d-4165ae50624f admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:40.081340 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377e1ea3-e74d-490c-934d-4165ae50624f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:44:40.081653 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377e1ea3-e74d-490c-934d-4165ae50624f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:40.083457 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377e1ea3-e74d-490c-934d-4165ae50624f admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:40.095991 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-3ea4850c-2f78-4fda-b46f-757debcd326f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99059a35e5994ef4a12dcd62447904a8'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:40.096918 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-3ea4850c-2f78-4fda-b46f-757debcd326f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99059a35e5994ef4a12dcd62447904a8'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:40.102187 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1055/2107] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:44:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:44:40.103931 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e1ea3-e74d-490c-934d-4165ae50624f admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:40.105489 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-377e1ea3-e74d-490c-934d-4165ae50624f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:40.116966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-3ea4850c-2f78-4fda-b46f-757debcd326f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99059a35e5994ef4a12dcd62447904a8'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:40.118422 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e1ea3-e74d-490c-934d-4165ae50624f admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:40.123852 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25efc10d-929d-4cf6-a583-eac217d10ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:40.124624 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25efc10d-929d-4cf6-a583-eac217d10ab5 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:40.125062 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25efc10d-929d-4cf6-a583-eac217d10ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:40.125330 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25efc10d-929d-4cf6-a583-eac217d10ab5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:40.126112 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25efc10d-929d-4cf6-a583-eac217d10ab5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:40.128348 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25efc10d-929d-4cf6-a583-eac217d10ab5 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:40.128788 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-3ea4850c-2f78-4fda-b46f-757debcd326f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99059a35e5994ef4a12dcd62447904a8'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:40.144710 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-3ea4850c-2f78-4fda-b46f-757debcd326f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99059a35e5994ef4a12dcd62447904a8'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:40.154019 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1053/2108] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:39 2022] DELETE /identity/v3/users/99059a35e5994ef4a12dcd62447904a8 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:40.158437 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:40.170383 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1056/2109] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:40 2022] POST /identity/v3/projects => generated 375 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:40.171633 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea4850c-2f78-4fda-b46f-757debcd326f admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:40.188036 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:40.267991 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea4850c-2f78-4fda-b46f-757debcd326f admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:40.338389 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0e1b4-27f8-484d-9a93-9f12a98fe9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:40.339112 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0e1b4-27f8-484d-9a93-9f12a98fe9c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:40.339402 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0e1b4-27f8-484d-9a93-9f12a98fe9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:40.339683 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0e1b4-27f8-484d-9a93-9f12a98fe9c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:40.340386 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac0e1b4-27f8-484d-9a93-9f12a98fe9c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:40.342480 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac0e1b4-27f8-484d-9a93-9f12a98fe9c6 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:40.367027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1054/2110] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:40 2022] POST /identity/v3/users => generated 288 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:44:40.372135 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25efc10d-929d-4cf6-a583-eac217d10ab5 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:40.384756 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25efc10d-929d-4cf6-a583-eac217d10ab5 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:40.391590 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680caa0-267f-42b6-8c9a-33f8acb74fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:40.392349 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680caa0-267f-42b6-8c9a-33f8acb74fad admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:40.392888 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680caa0-267f-42b6-8c9a-33f8acb74fad admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:40.393243 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680caa0-267f-42b6-8c9a-33f8acb74fad admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:40.393984 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b680caa0-267f-42b6-8c9a-33f8acb74fad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:40.395724 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b680caa0-267f-42b6-8c9a-33f8acb74fad admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:40.416134 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1057/2111] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:40.419655 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0e1b4-27f8-484d-9a93-9f12a98fe9c6 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:40.439923 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0e1b4-27f8-484d-9a93-9f12a98fe9c6 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:40.449884 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d46dd7-9a64-48a0-ae83-bec594656b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:40.450817 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d46dd7-9a64-48a0-ae83-bec594656b47 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:40.451144 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d46dd7-9a64-48a0-ae83-bec594656b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:40.451229 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d46dd7-9a64-48a0-ae83-bec594656b47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:40.452067 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d46dd7-9a64-48a0-ae83-bec594656b47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:40.453844 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d46dd7-9a64-48a0-ae83-bec594656b47 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:40.469875 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1055/2112] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:40.471953 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680caa0-267f-42b6-8c9a-33f8acb74fad admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:40.482915 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680caa0-267f-42b6-8c9a-33f8acb74fad admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:40.488286 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ff0a2-b318-4d74-8fa8-dd948dc7142d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:40.489102 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ff0a2-b318-4d74-8fa8-dd948dc7142d admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:40.489467 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ff0a2-b318-4d74-8fa8-dd948dc7142d admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:40.491023 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ff0a2-b318-4d74-8fa8-dd948dc7142d admin admin] PATH_INFO: `/v3/projects/fae043d79ce040e0affb39e23162523a/users/453c056ef8674822a0fc93c1c6675664/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:40.522375 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7ff0a2-b318-4d74-8fa8-dd948dc7142d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fae043d79ce040e0affb39e23162523a, user_id=453c056ef8674822a0fc93c1c6675664, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:40.525159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7ff0a2-b318-4d74-8fa8-dd948dc7142d admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:40.541126 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1058/2113] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:40 2022] PUT /identity/v3/projects/fae043d79ce040e0affb39e23162523a/users/453c056ef8674822a0fc93c1c6675664/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:40.552881 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab7d42-784a-4bf6-8d94-8187d652574f None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:40.553173 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab7d42-784a-4bf6-8d94-8187d652574f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:40.553455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab7d42-784a-4bf6-8d94-8187d652574f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:40.599888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-deab7d42-784a-4bf6-8d94-8187d652574f None None] MFA Rules not processed for user `453c056ef8674822a0fc93c1c6675664`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:40.599888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-deab7d42-784a-4bf6-8d94-8187d652574f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:40.601201 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ff0a2-b318-4d74-8fa8-dd948dc7142d admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:40.605678 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5b7ff0a2-b318-4d74-8fa8-dd948dc7142d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:40.699936 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ff0a2-b318-4d74-8fa8-dd948dc7142d admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:40.705389 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a9861-0871-49bb-96cf-68836751b8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:40.706257 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a9861-0871-49bb-96cf-68836751b8d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:40.706538 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a9861-0871-49bb-96cf-68836751b8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:40.706820 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a9861-0871-49bb-96cf-68836751b8d1 admin admin] PATH_INFO: `/v3/projects/da98037792874fab89ab34e8f9b25aef` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:40.710276 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809a9861-0871-49bb-96cf-68836751b8d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da98037792874fab89ab34e8f9b25aef)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:40.712235 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809a9861-0871-49bb-96cf-68836751b8d1 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:40.719708 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1056/2114] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:40.732056 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab7d42-784a-4bf6-8d94-8187d652574f None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:40.733424 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-deab7d42-784a-4bf6-8d94-8187d652574f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:40.762913 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-809a9861-0871-49bb-96cf-68836751b8d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da98037792874fab89ab34e8f9b25aef', 'request_id': 'req-809a9861-0871-49bb-96cf-68836751b8d1'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:40.768713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:40.775860 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-809a9861-0871-49bb-96cf-68836751b8d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da98037792874fab89ab34e8f9b25aef', 'request_id': 'req-809a9861-0871-49bb-96cf-68836751b8d1'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:40.783367 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-809a9861-0871-49bb-96cf-68836751b8d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da98037792874fab89ab34e8f9b25aef', 'request_id': 'req-809a9861-0871-49bb-96cf-68836751b8d1'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:40.804354 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab7d42-784a-4bf6-8d94-8187d652574f None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:40.805912 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-809a9861-0871-49bb-96cf-68836751b8d1 admin admin] The token cache is being invalidate because project da98037792874fab89ab34e8f9b25aef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:40.806303 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-809a9861-0871-49bb-96cf-68836751b8d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:40.806966 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-809a9861-0871-49bb-96cf-68836751b8d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:40.808799 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1059/2115] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:40 2022] DELETE /identity/v3/projects/da98037792874fab89ab34e8f9b25aef => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:40.813736 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d775f4-90a5-41d3-b64a-3af4a51b3206 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:40.814086 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d775f4-90a5-41d3-b64a-3af4a51b3206 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:40.814495 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d775f4-90a5-41d3-b64a-3af4a51b3206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:40.838128 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-c3d775f4-90a5-41d3-b64a-3af4a51b3206 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:40.839470 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c3d775f4-90a5-41d3-b64a-3af4a51b3206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:40.865543 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f6ee9c-e097-45b2-82be-ac8831a0229f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:40.866209 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6ee9c-e097-45b2-82be-ac8831a0229f service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:40.866503 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6ee9c-e097-45b2-82be-ac8831a0229f service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:40.866786 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6ee9c-e097-45b2-82be-ac8831a0229f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:40.869006 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c3f6ee9c-e097-45b2-82be-ac8831a0229f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:40.894509 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:40.912379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1060/2116] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:44:40.921253 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e968c-4b05-4ef8-a870-fe43339f0a08 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:40.921598 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e968c-4b05-4ef8-a870-fe43339f0a08 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:40.921884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e968c-4b05-4ef8-a870-fe43339f0a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:40.941307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f6ee9c-e097-45b2-82be-ac8831a0229f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:40.943398 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f6ee9c-e097-45b2-82be-ac8831a0229f service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:40.945050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-601e968c-4b05-4ef8-a870-fe43339f0a08 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:40.946270 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-601e968c-4b05-4ef8-a870-fe43339f0a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:40.953671 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1057/2117] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:40.955817 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1061/2118] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:44:40.960475 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f6ee9c-e097-45b2-82be-ac8831a0229f service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:40.971232 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f6ee9c-e097-45b2-82be-ac8831a0229f service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:40.975854 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157d04e-e506-4c19-8d12-6f9eb758c9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:40.976511 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157d04e-e506-4c19-8d12-6f9eb758c9a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:40.976792 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157d04e-e506-4c19-8d12-6f9eb758c9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:40.977067 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157d04e-e506-4c19-8d12-6f9eb758c9a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:40.977787 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8157d04e-e506-4c19-8d12-6f9eb758c9a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:44:40.978093 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8157d04e-e506-4c19-8d12-6f9eb758c9a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:40.980104 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8157d04e-e506-4c19-8d12-6f9eb758c9a3 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:40.992946 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1058/2119] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:44:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:44:40.997392 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601e968c-4b05-4ef8-a870-fe43339f0a08 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:41.007731 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601e968c-4b05-4ef8-a870-fe43339f0a08 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:41.012610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ba9a7-fd9b-451d-804b-c4fb88a35fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:41.013294 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ba9a7-fd9b-451d-804b-c4fb88a35fd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:41.013583 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ba9a7-fd9b-451d-804b-c4fb88a35fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:41.013866 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ba9a7-fd9b-451d-804b-c4fb88a35fd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:41.014587 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5ba9a7-fd9b-451d-804b-c4fb88a35fd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:41.016647 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5ba9a7-fd9b-451d-804b-c4fb88a35fd5 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:41.047035 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1062/2120] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:40 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:41.049245 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157d04e-e506-4c19-8d12-6f9eb758c9a3 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:41.059917 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157d04e-e506-4c19-8d12-6f9eb758c9a3 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:41.127214 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a0abbb-d4ec-4c74-a426-2a97cde9a226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:41.127924 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0abbb-d4ec-4c74-a426-2a97cde9a226 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:41.128545 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0abbb-d4ec-4c74-a426-2a97cde9a226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:41.128545 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0abbb-d4ec-4c74-a426-2a97cde9a226 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:41.129275 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a0abbb-d4ec-4c74-a426-2a97cde9a226 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:41.132014 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a0abbb-d4ec-4c74-a426-2a97cde9a226 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:41.163028 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1059/2121] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:41 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:41.165594 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ba9a7-fd9b-451d-804b-c4fb88a35fd5 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:41.176356 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ba9a7-fd9b-451d-804b-c4fb88a35fd5 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:41.183518 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08c15c-8dba-4778-bd2c-ac4ccc327b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:41.183518 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08c15c-8dba-4778-bd2c-ac4ccc327b3f admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:41.183914 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08c15c-8dba-4778-bd2c-ac4ccc327b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:41.184013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08c15c-8dba-4778-bd2c-ac4ccc327b3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:41.185100 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e08c15c-8dba-4778-bd2c-ac4ccc327b3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:41.186576 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e08c15c-8dba-4778-bd2c-ac4ccc327b3f admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:41.200203 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1063/2122] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:41.204584 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a0abbb-d4ec-4c74-a426-2a97cde9a226 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:41.217715 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a0abbb-d4ec-4c74-a426-2a97cde9a226 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:41.223376 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26730f0d-dfa1-44d8-8083-f6be329f156d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:41.224614 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26730f0d-dfa1-44d8-8083-f6be329f156d admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:41.225174 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26730f0d-dfa1-44d8-8083-f6be329f156d admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:41.225663 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26730f0d-dfa1-44d8-8083-f6be329f156d admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:41.227055 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26730f0d-dfa1-44d8-8083-f6be329f156d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:41.229636 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26730f0d-dfa1-44d8-8083-f6be329f156d admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:41.251068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1060/2123] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:41.256205 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08c15c-8dba-4778-bd2c-ac4ccc327b3f admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:41.267863 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08c15c-8dba-4778-bd2c-ac4ccc327b3f admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:41.275458 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd82d0-6711-48f2-8172-6b48089c3d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:41.276491 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd82d0-6711-48f2-8172-6b48089c3d45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:41.276812 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd82d0-6711-48f2-8172-6b48089c3d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:41.277629 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd82d0-6711-48f2-8172-6b48089c3d45 admin admin] PATH_INFO: `/v3/projects/8ec4b30b1bc14ab3b932ca26534acd48/users/dbede9987a0448e2b5d446f2495eae64/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:41.297249 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd82d0-6711-48f2-8172-6b48089c3d45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ec4b30b1bc14ab3b932ca26534acd48, user_id=dbede9987a0448e2b5d446f2495eae64, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:41.300243 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd82d0-6711-48f2-8172-6b48089c3d45 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:41.335093 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1064/2124] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:41 2022] PUT /identity/v3/projects/8ec4b30b1bc14ab3b932ca26534acd48/users/dbede9987a0448e2b5d446f2495eae64/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:41.347535 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a3eed-f36d-4cfe-a5f1-6384a033dc59 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:41.348091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a3eed-f36d-4cfe-a5f1-6384a033dc59 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:41.348091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a3eed-f36d-4cfe-a5f1-6384a033dc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:41.353288 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:41.394587 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-c38a3eed-f36d-4cfe-a5f1-6384a033dc59 None None] MFA Rules not processed for user `dbede9987a0448e2b5d446f2495eae64`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:41.396060 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c38a3eed-f36d-4cfe-a5f1-6384a033dc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:41.504730 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1061/2125] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:41.509763 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd82d0-6711-48f2-8172-6b48089c3d45 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:41.675372 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd82d0-6711-48f2-8172-6b48089c3d45 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:41.684167 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a488677d-f641-43c4-96ba-a551368a457b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:41.684862 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a488677d-f641-43c4-96ba-a551368a457b admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:41.685177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a488677d-f641-43c4-96ba-a551368a457b admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:41.685544 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a488677d-f641-43c4-96ba-a551368a457b admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:41.686258 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a488677d-f641-43c4-96ba-a551368a457b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:41.688264 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a488677d-f641-43c4-96ba-a551368a457b admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:41.726932 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1065/2126] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:41 2022] POST /identity/v3/projects => generated 387 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:41.730825 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a3eed-f36d-4cfe-a5f1-6384a033dc59 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:41.755397 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a3eed-f36d-4cfe-a5f1-6384a033dc59 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:41.852828 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b627c0-b430-404e-a4c9-fb3eaa621be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:41.853676 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b627c0-b430-404e-a4c9-fb3eaa621be7 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:41.855262 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b627c0-b430-404e-a4c9-fb3eaa621be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:41.855262 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b627c0-b430-404e-a4c9-fb3eaa621be7 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:41.855567 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b627c0-b430-404e-a4c9-fb3eaa621be7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:41.857571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b627c0-b430-404e-a4c9-fb3eaa621be7 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:41.884939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1062/2127] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:41 2022] POST /identity/v3/users => generated 300 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:41.893223 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a488677d-f641-43c4-96ba-a551368a457b admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:41.905762 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a488677d-f641-43c4-96ba-a551368a457b admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:41.911386 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e2d3f-d340-40a8-a495-b72bfa11ef77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:41.912520 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e2d3f-d340-40a8-a495-b72bfa11ef77 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:41.912901 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e2d3f-d340-40a8-a495-b72bfa11ef77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:41.913359 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e2d3f-d340-40a8-a495-b72bfa11ef77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:41.914210 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e2d3f-d340-40a8-a495-b72bfa11ef77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:41.916106 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e2d3f-d340-40a8-a495-b72bfa11ef77 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:41.928618 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1066/2128] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:44:41.933939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b627c0-b430-404e-a4c9-fb3eaa621be7 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:41.945124 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b627c0-b430-404e-a4c9-fb3eaa621be7 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:41.951291 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e0b10-1e02-4aed-833d-9b90a88b278b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:41.951649 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e0b10-1e02-4aed-833d-9b90a88b278b admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:41.951931 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e0b10-1e02-4aed-833d-9b90a88b278b admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:41.952208 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e0b10-1e02-4aed-833d-9b90a88b278b admin admin] PATH_INFO: `/v3/projects/f611e242cfdb409db923e73bc73026f3/users/84edb2980b0a44378c3c35d5281ac7e0/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:41.969215 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708e0b10-1e02-4aed-833d-9b90a88b278b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f611e242cfdb409db923e73bc73026f3, user_id=84edb2980b0a44378c3c35d5281ac7e0, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:41.971936 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708e0b10-1e02-4aed-833d-9b90a88b278b admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:41.994098 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1063/2129] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:41 2022] PUT /identity/v3/projects/f611e242cfdb409db923e73bc73026f3/users/84edb2980b0a44378c3c35d5281ac7e0/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:41.995596 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e2d3f-d340-40a8-a495-b72bfa11ef77 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:42.008177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e2d3f-d340-40a8-a495-b72bfa11ef77 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:42.029699 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:42.103990 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255582f-68c7-49d0-b055-59fedaf99fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:42.104774 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255582f-68c7-49d0-b055-59fedaf99fba admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:42.104974 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255582f-68c7-49d0-b055-59fedaf99fba admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:42.105183 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255582f-68c7-49d0-b055-59fedaf99fba admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:42.105928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f255582f-68c7-49d0-b055-59fedaf99fba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:42.108311 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f255582f-68c7-49d0-b055-59fedaf99fba admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:42.122470 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1067/2130] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:42.126096 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e0b10-1e02-4aed-833d-9b90a88b278b admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:42.142055 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e0b10-1e02-4aed-833d-9b90a88b278b admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:42.148856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90ef57-1b0e-4db3-ba16-1fbcf34079e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:42.149788 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90ef57-1b0e-4db3-ba16-1fbcf34079e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:42.150073 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90ef57-1b0e-4db3-ba16-1fbcf34079e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:42.150347 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90ef57-1b0e-4db3-ba16-1fbcf34079e3 admin admin] PATH_INFO: `/v3/projects/f611e242cfdb409db923e73bc73026f3/users/84edb2980b0a44378c3c35d5281ac7e0/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:42.152851 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c90ef57-1b0e-4db3-ba16-1fbcf34079e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f611e242cfdb409db923e73bc73026f3, user_id=84edb2980b0a44378c3c35d5281ac7e0, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:42.155651 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c90ef57-1b0e-4db3-ba16-1fbcf34079e3 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:42.180304 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1064/2131] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:42 2022] PUT /identity/v3/projects/f611e242cfdb409db923e73bc73026f3/users/84edb2980b0a44378c3c35d5281ac7e0/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:42.182541 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e646d-5900-4832-83ef-a683843f18a5 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:42.182911 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e646d-5900-4832-83ef-a683843f18a5 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:42.183298 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e646d-5900-4832-83ef-a683843f18a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:42.222507 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-254e646d-5900-4832-83ef-a683843f18a5 None None] MFA Rules not processed for user `84edb2980b0a44378c3c35d5281ac7e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:42.224884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-254e646d-5900-4832-83ef-a683843f18a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:42.323441 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1068/2132] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:44:42.361812 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90ef57-1b0e-4db3-ba16-1fbcf34079e3 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:42.363935 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-6c90ef57-1b0e-4db3-ba16-1fbcf34079e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:42.388733 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:42.464514 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90ef57-1b0e-4db3-ba16-1fbcf34079e3 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:42.468450 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32a667-2df1-44ac-bed1-3e14ec91322f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:42.469140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32a667-2df1-44ac-bed1-3e14ec91322f service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:42.469425 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32a667-2df1-44ac-bed1-3e14ec91322f service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:42.469707 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32a667-2df1-44ac-bed1-3e14ec91322f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:42.480243 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c32a667-2df1-44ac-bed1-3e14ec91322f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:42.481706 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c32a667-2df1-44ac-bed1-3e14ec91322f service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:42.493566 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1065/2133] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:42.523550 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e646d-5900-4832-83ef-a683843f18a5 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:42.615699 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e646d-5900-4832-83ef-a683843f18a5 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:42.622066 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1cc5a-bfe8-45b8-92f9-b7736c657461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:42.623428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1cc5a-bfe8-45b8-92f9-b7736c657461 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:42.623896 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1cc5a-bfe8-45b8-92f9-b7736c657461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:42.624433 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1cc5a-bfe8-45b8-92f9-b7736c657461 admin admin] PATH_INFO: `/v3/users/dbede9987a0448e2b5d446f2495eae64` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:42.626053 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a1cc5a-bfe8-45b8-92f9-b7736c657461 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbede9987a0448e2b5d446f2495eae64)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:42.629031 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a1cc5a-bfe8-45b8-92f9-b7736c657461 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:42.689890 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:42.704042 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-07a1cc5a-bfe8-45b8-92f9-b7736c657461 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbede9987a0448e2b5d446f2495eae64'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:42.705022 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-07a1cc5a-bfe8-45b8-92f9-b7736c657461 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbede9987a0448e2b5d446f2495eae64'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:42.721846 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-07a1cc5a-bfe8-45b8-92f9-b7736c657461 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbede9987a0448e2b5d446f2495eae64'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:42.728146 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-07a1cc5a-bfe8-45b8-92f9-b7736c657461 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbede9987a0448e2b5d446f2495eae64'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:42.736469 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-07a1cc5a-bfe8-45b8-92f9-b7736c657461 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbede9987a0448e2b5d446f2495eae64'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:42.751936 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1069/2134] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:42 2022] DELETE /identity/v3/users/dbede9987a0448e2b5d446f2495eae64 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:42.761531 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b0f2b-1446-48a3-bd13-44b3887b0f87 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:42.762421 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b0f2b-1446-48a3-bd13-44b3887b0f87 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:42.762521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b0f2b-1446-48a3-bd13-44b3887b0f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:42.808808 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-8d9b0f2b-1446-48a3-bd13-44b3887b0f87 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:42.810158 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-8d9b0f2b-1446-48a3-bd13-44b3887b0f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:42.925409 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1066/2135] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:42.936691 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1cc5a-bfe8-45b8-92f9-b7736c657461 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:42.938572 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-07a1cc5a-bfe8-45b8-92f9-b7736c657461 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:43.056817 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1cc5a-bfe8-45b8-92f9-b7736c657461 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:43.060817 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72bf491-52d9-4ada-bf19-0cb029be0d77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:43.061474 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72bf491-52d9-4ada-bf19-0cb029be0d77 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:43.061760 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72bf491-52d9-4ada-bf19-0cb029be0d77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:43.062059 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72bf491-52d9-4ada-bf19-0cb029be0d77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:43.075416 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72bf491-52d9-4ada-bf19-0cb029be0d77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:43.076919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72bf491-52d9-4ada-bf19-0cb029be0d77 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:43.088733 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1070/2136] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:43.403674 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9b0f2b-1446-48a3-bd13-44b3887b0f87 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:43.405316 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-8d9b0f2b-1446-48a3-bd13-44b3887b0f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:43.428160 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9b0f2b-1446-48a3-bd13-44b3887b0f87 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:43.433644 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dbf70e-16cb-4ee2-a5ad-17fb135677f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:43.434677 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbf70e-16cb-4ee2-a5ad-17fb135677f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:43.435061 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbf70e-16cb-4ee2-a5ad-17fb135677f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:43.435415 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbf70e-16cb-4ee2-a5ad-17fb135677f9 admin admin] PATH_INFO: `/v3/projects/8ec4b30b1bc14ab3b932ca26534acd48` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:43.437804 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dbf70e-16cb-4ee2-a5ad-17fb135677f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ec4b30b1bc14ab3b932ca26534acd48)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:43.440297 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dbf70e-16cb-4ee2-a5ad-17fb135677f9 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:43.443868 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761529c-169e-40d6-a511-48f9324b646b None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:43.444168 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761529c-169e-40d6-a511-48f9324b646b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:43.444506 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761529c-169e-40d6-a511-48f9324b646b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:43.466433 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-f761529c-169e-40d6-a511-48f9324b646b None None] MFA Rules not processed for user `453c056ef8674822a0fc93c1c6675664`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:43.467785 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f761529c-169e-40d6-a511-48f9324b646b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:43.505367 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-49dbf70e-16cb-4ee2-a5ad-17fb135677f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ec4b30b1bc14ab3b932ca26534acd48', 'request_id': 'req-49dbf70e-16cb-4ee2-a5ad-17fb135677f9'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:43.514964 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-49dbf70e-16cb-4ee2-a5ad-17fb135677f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ec4b30b1bc14ab3b932ca26534acd48', 'request_id': 'req-49dbf70e-16cb-4ee2-a5ad-17fb135677f9'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:43.523200 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-49dbf70e-16cb-4ee2-a5ad-17fb135677f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ec4b30b1bc14ab3b932ca26534acd48', 'request_id': 'req-49dbf70e-16cb-4ee2-a5ad-17fb135677f9'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:43.534690 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-49dbf70e-16cb-4ee2-a5ad-17fb135677f9 admin admin] The token cache is being invalidate because project 8ec4b30b1bc14ab3b932ca26534acd48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:43.534967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-49dbf70e-16cb-4ee2-a5ad-17fb135677f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:43.535707 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-49dbf70e-16cb-4ee2-a5ad-17fb135677f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:43.537952 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1067/2137] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:43 2022] DELETE /identity/v3/projects/8ec4b30b1bc14ab3b932ca26534acd48 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:43.542054 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dbf70e-16cb-4ee2-a5ad-17fb135677f9 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:43.543513 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-49dbf70e-16cb-4ee2-a5ad-17fb135677f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:43.547848 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1071/2138] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:43.573263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:43.622734 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dbf70e-16cb-4ee2-a5ad-17fb135677f9 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:43.629427 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1f425c-df46-46b4-b02a-ae3c9b6de16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:43.631597 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f425c-df46-46b4-b02a-ae3c9b6de16f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:43.632200 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f425c-df46-46b4-b02a-ae3c9b6de16f admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:43.632704 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f425c-df46-46b4-b02a-ae3c9b6de16f admin admin] PATH_INFO: `/v3/users/84edb2980b0a44378c3c35d5281ac7e0` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:43.635132 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1f425c-df46-46b4-b02a-ae3c9b6de16f admin admin] RBAC: Authorizing `identity:delete_user(user_id=84edb2980b0a44378c3c35d5281ac7e0)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:43.638985 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1f425c-df46-46b4-b02a-ae3c9b6de16f admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:43.697816 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0b1f425c-df46-46b4-b02a-ae3c9b6de16f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84edb2980b0a44378c3c35d5281ac7e0'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:43.698433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0b1f425c-df46-46b4-b02a-ae3c9b6de16f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84edb2980b0a44378c3c35d5281ac7e0'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:43.713545 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0b1f425c-df46-46b4-b02a-ae3c9b6de16f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84edb2980b0a44378c3c35d5281ac7e0'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:43.722483 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0b1f425c-df46-46b4-b02a-ae3c9b6de16f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84edb2980b0a44378c3c35d5281ac7e0'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:43.731202 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0b1f425c-df46-46b4-b02a-ae3c9b6de16f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84edb2980b0a44378c3c35d5281ac7e0'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:43.737953 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1068/2139] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:43 2022] DELETE /identity/v3/users/84edb2980b0a44378c3c35d5281ac7e0 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:44.074051 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761529c-169e-40d6-a511-48f9324b646b None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:44.093382 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761529c-169e-40d6-a511-48f9324b646b None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:44.221911 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:44.249117 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e78ca-eed9-4805-ac9c-ec9e681a9c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:44.250920 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e78ca-eed9-4805-ac9c-ec9e681a9c91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:44.252290 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e78ca-eed9-4805-ac9c-ec9e681a9c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:44.252290 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e78ca-eed9-4805-ac9c-ec9e681a9c91 admin admin] PATH_INFO: `/v3/projects/f611e242cfdb409db923e73bc73026f3` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:44.254086 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e78ca-eed9-4805-ac9c-ec9e681a9c91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f611e242cfdb409db923e73bc73026f3)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:44.257884 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e78ca-eed9-4805-ac9c-ec9e681a9c91 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:44.359260 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b84e78ca-eed9-4805-ac9c-ec9e681a9c91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f611e242cfdb409db923e73bc73026f3', 'request_id': 'req-b84e78ca-eed9-4805-ac9c-ec9e681a9c91'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:44.371739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b84e78ca-eed9-4805-ac9c-ec9e681a9c91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f611e242cfdb409db923e73bc73026f3', 'request_id': 'req-b84e78ca-eed9-4805-ac9c-ec9e681a9c91'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:44.381099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b84e78ca-eed9-4805-ac9c-ec9e681a9c91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f611e242cfdb409db923e73bc73026f3', 'request_id': 'req-b84e78ca-eed9-4805-ac9c-ec9e681a9c91'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:44.392983 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b84e78ca-eed9-4805-ac9c-ec9e681a9c91 admin admin] The token cache is being invalidate because project f611e242cfdb409db923e73bc73026f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:44.393265 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b84e78ca-eed9-4805-ac9c-ec9e681a9c91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:44.393966 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b84e78ca-eed9-4805-ac9c-ec9e681a9c91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:44.400506 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1072/2140] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:44 2022] DELETE /identity/v3/projects/f611e242cfdb409db923e73bc73026f3 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:44.406008 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd67b27-7e7b-4f94-b73f-a24998d582c8 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:44.406507 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd67b27-7e7b-4f94-b73f-a24998d582c8 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:44.406739 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd67b27-7e7b-4f94-b73f-a24998d582c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:44.456572 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-1fd67b27-7e7b-4f94-b73f-a24998d582c8 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:44.457800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1fd67b27-7e7b-4f94-b73f-a24998d582c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:44.542886 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1069/2141] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:44.550745 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea88d9-9ad4-44e2-b295-ad4690ba741c None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:44.551270 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea88d9-9ad4-44e2-b295-ad4690ba741c None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:44.551369 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea88d9-9ad4-44e2-b295-ad4690ba741c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:44.573267 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-7cea88d9-9ad4-44e2-b295-ad4690ba741c None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:44.575338 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-7cea88d9-9ad4-44e2-b295-ad4690ba741c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:44.585257 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1073/2142] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:44.589602 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd67b27-7e7b-4f94-b73f-a24998d582c8 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:44.600165 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd67b27-7e7b-4f94-b73f-a24998d582c8 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:44.605418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c52104-e135-4dbe-a494-d31f9e934f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:44.606262 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c52104-e135-4dbe-a494-d31f9e934f48 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:44.606506 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c52104-e135-4dbe-a494-d31f9e934f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:44.607027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c52104-e135-4dbe-a494-d31f9e934f48 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:44.607735 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c52104-e135-4dbe-a494-d31f9e934f48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:44:44.608103 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c52104-e135-4dbe-a494-d31f9e934f48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:44.610100 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c52104-e135-4dbe-a494-d31f9e934f48 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:44.627031 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1070/2143] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:44:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:44:44.630003 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea88d9-9ad4-44e2-b295-ad4690ba741c None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:44.639779 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea88d9-9ad4-44e2-b295-ad4690ba741c None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:44.645834 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f1484-10bd-450a-8f74-ba34ceadbd57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:44.646779 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f1484-10bd-450a-8f74-ba34ceadbd57 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:44.646910 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f1484-10bd-450a-8f74-ba34ceadbd57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:44.647222 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f1484-10bd-450a-8f74-ba34ceadbd57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:44.647951 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48f1484-10bd-450a-8f74-ba34ceadbd57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:44.649883 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48f1484-10bd-450a-8f74-ba34ceadbd57 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:44.681943 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1074/2144] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:44 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:44.685997 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c52104-e135-4dbe-a494-d31f9e934f48 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:44.697024 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c52104-e135-4dbe-a494-d31f9e934f48 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:44.707916 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:44.758812 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77191145-4b25-433d-8251-1ac7b6f1c89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:44.759561 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77191145-4b25-433d-8251-1ac7b6f1c89f admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:44.759848 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77191145-4b25-433d-8251-1ac7b6f1c89f admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:44.760121 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77191145-4b25-433d-8251-1ac7b6f1c89f admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:44.760821 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77191145-4b25-433d-8251-1ac7b6f1c89f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:44.762737 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77191145-4b25-433d-8251-1ac7b6f1c89f admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:44.786781 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1071/2145] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:44 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:44.788494 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f1484-10bd-450a-8f74-ba34ceadbd57 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:44.798986 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f1484-10bd-450a-8f74-ba34ceadbd57 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:44.804134 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c33300c-c459-43d5-852b-bf4a4d3c694c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:44.804818 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33300c-c459-43d5-852b-bf4a4d3c694c admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:44.805106 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33300c-c459-43d5-852b-bf4a4d3c694c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:44.805383 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33300c-c459-43d5-852b-bf4a4d3c694c admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:44.806119 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c33300c-c459-43d5-852b-bf4a4d3c694c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:44.807822 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c33300c-c459-43d5-852b-bf4a4d3c694c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:44.818619 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1075/2146] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:44.822947 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77191145-4b25-433d-8251-1ac7b6f1c89f admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:44.836233 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77191145-4b25-433d-8251-1ac7b6f1c89f admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:44.840997 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93ee9c-00d8-4e6b-8099-279f055c5f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:44.841656 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93ee9c-00d8-4e6b-8099-279f055c5f06 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:44.841940 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93ee9c-00d8-4e6b-8099-279f055c5f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:44.842218 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93ee9c-00d8-4e6b-8099-279f055c5f06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:44.842943 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa93ee9c-00d8-4e6b-8099-279f055c5f06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:44.844576 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa93ee9c-00d8-4e6b-8099-279f055c5f06 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:44.855573 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1072/2147] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:44.859520 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c33300c-c459-43d5-852b-bf4a4d3c694c admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:44.869445 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c33300c-c459-43d5-852b-bf4a4d3c694c admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:44.874005 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccde2ff-c6fc-42ca-8d53-b58686de3200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:44.874921 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccde2ff-c6fc-42ca-8d53-b58686de3200 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:44.875231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccde2ff-c6fc-42ca-8d53-b58686de3200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:44.875507 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccde2ff-c6fc-42ca-8d53-b58686de3200 admin admin] PATH_INFO: `/v3/projects/1c49db0fb633401c85b2875600a04408/users/9d29a989cf0848b4936b059b0c281098/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:44.889859 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccde2ff-c6fc-42ca-8d53-b58686de3200 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c49db0fb633401c85b2875600a04408, user_id=9d29a989cf0848b4936b059b0c281098, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:44.892544 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccde2ff-c6fc-42ca-8d53-b58686de3200 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:44.908254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1076/2148] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:44 2022] PUT /identity/v3/projects/1c49db0fb633401c85b2875600a04408/users/9d29a989cf0848b4936b059b0c281098/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:44.919053 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3745d-3f9e-46c6-8b07-13deafb72cdb None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:44.919351 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3745d-3f9e-46c6-8b07-13deafb72cdb None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:44.919633 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3745d-3f9e-46c6-8b07-13deafb72cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:44.948246 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-85f3745d-3f9e-46c6-8b07-13deafb72cdb None None] MFA Rules not processed for user `9d29a989cf0848b4936b059b0c281098`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:44.949527 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-85f3745d-3f9e-46c6-8b07-13deafb72cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:45.021540 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1073/2149] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:45.026132 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccde2ff-c6fc-42ca-8d53-b58686de3200 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:45.087718 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccde2ff-c6fc-42ca-8d53-b58686de3200 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:45.092358 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e834f66-54cf-4068-938e-1c22ab7df695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:45.093023 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e834f66-54cf-4068-938e-1c22ab7df695 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:45.093307 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e834f66-54cf-4068-938e-1c22ab7df695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:45.093586 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e834f66-54cf-4068-938e-1c22ab7df695 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:45.094288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e834f66-54cf-4068-938e-1c22ab7df695 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:45.096234 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e834f66-54cf-4068-938e-1c22ab7df695 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:45.109697 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:45.123306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1077/2150] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:45 2022] POST /identity/v3/projects => generated 381 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:45.127342 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3745d-3f9e-46c6-8b07-13deafb72cdb None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:45.137908 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3745d-3f9e-46c6-8b07-13deafb72cdb None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:45.194237 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f7368-98f7-42c5-8678-25d45c98d4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:45.194903 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f7368-98f7-42c5-8678-25d45c98d4df admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:45.195414 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f7368-98f7-42c5-8678-25d45c98d4df admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:45.195699 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f7368-98f7-42c5-8678-25d45c98d4df admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:45.196401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f7368-98f7-42c5-8678-25d45c98d4df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:45.198320 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f7368-98f7-42c5-8678-25d45c98d4df admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:45.217499 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1074/2151] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:45 2022] POST /identity/v3/users => generated 297 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:45.221634 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e834f66-54cf-4068-938e-1c22ab7df695 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:45.232013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e834f66-54cf-4068-938e-1c22ab7df695 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:45.236586 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099008f5-3c17-4a1c-9391-8677be25bde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:45.237232 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099008f5-3c17-4a1c-9391-8677be25bde5 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:45.237512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099008f5-3c17-4a1c-9391-8677be25bde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:45.237789 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099008f5-3c17-4a1c-9391-8677be25bde5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:45.238606 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099008f5-3c17-4a1c-9391-8677be25bde5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:45.240650 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099008f5-3c17-4a1c-9391-8677be25bde5 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:45.251634 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1078/2152] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:45.256346 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f7368-98f7-42c5-8678-25d45c98d4df admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:45.267445 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f7368-98f7-42c5-8678-25d45c98d4df admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:45.273010 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee76de39-c1b7-43a8-b321-8015ccf454e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:45.273881 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76de39-c1b7-43a8-b321-8015ccf454e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:45.274166 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76de39-c1b7-43a8-b321-8015ccf454e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:45.274446 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76de39-c1b7-43a8-b321-8015ccf454e3 admin admin] PATH_INFO: `/v3/projects/ad5c2864ddfe424e8ec446ee69fab016/users/32b777358749425da8cea0e9804ddc3d/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:45.283640 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:45.297307 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee76de39-c1b7-43a8-b321-8015ccf454e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad5c2864ddfe424e8ec446ee69fab016, user_id=32b777358749425da8cea0e9804ddc3d, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:45.300022 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee76de39-c1b7-43a8-b321-8015ccf454e3 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:45.321297 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1075/2153] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:45 2022] PUT /identity/v3/projects/ad5c2864ddfe424e8ec446ee69fab016/users/32b777358749425da8cea0e9804ddc3d/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:45.325288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099008f5-3c17-4a1c-9391-8677be25bde5 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:45.337332 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099008f5-3c17-4a1c-9391-8677be25bde5 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:45.426396 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaeeb95-8967-4cd3-9bc5-ff1793d51f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:45.427536 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaeeb95-8967-4cd3-9bc5-ff1793d51f53 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:45.428045 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaeeb95-8967-4cd3-9bc5-ff1793d51f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:45.428562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaeeb95-8967-4cd3-9bc5-ff1793d51f53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:45.429384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaeeb95-8967-4cd3-9bc5-ff1793d51f53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:45.431117 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaeeb95-8967-4cd3-9bc5-ff1793d51f53 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:45.445279 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1079/2154] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:45 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:45.447146 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee76de39-c1b7-43a8-b321-8015ccf454e3 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:45.460348 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee76de39-c1b7-43a8-b321-8015ccf454e3 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:45.465650 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d32c5c-988d-45d6-a043-f57e84850944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:45.466701 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d32c5c-988d-45d6-a043-f57e84850944 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:45.466985 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d32c5c-988d-45d6-a043-f57e84850944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:45.467354 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d32c5c-988d-45d6-a043-f57e84850944 admin admin] PATH_INFO: `/v3/projects/ad5c2864ddfe424e8ec446ee69fab016/users/32b777358749425da8cea0e9804ddc3d/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:45.469746 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d32c5c-988d-45d6-a043-f57e84850944 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad5c2864ddfe424e8ec446ee69fab016, user_id=32b777358749425da8cea0e9804ddc3d, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:45.472986 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d32c5c-988d-45d6-a043-f57e84850944 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:45.484970 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1076/2155] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:45 2022] PUT /identity/v3/projects/ad5c2864ddfe424e8ec446ee69fab016/users/32b777358749425da8cea0e9804ddc3d/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:45.500762 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e3ab1-09e4-4070-8243-608e646ee924 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:45.501296 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e3ab1-09e4-4070-8243-608e646ee924 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:45.501656 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e3ab1-09e4-4070-8243-608e646ee924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:45.538054 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-543e3ab1-09e4-4070-8243-608e646ee924 None None] MFA Rules not processed for user `32b777358749425da8cea0e9804ddc3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:45.539611 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-543e3ab1-09e4-4070-8243-608e646ee924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:45.657018 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1080/2156] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:45 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:45.668884 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d32c5c-988d-45d6-a043-f57e84850944 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:45.672179 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-30d32c5c-988d-45d6-a043-f57e84850944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:45.763820 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d32c5c-988d-45d6-a043-f57e84850944 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:45.768910 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36247c81-dc69-4668-a536-812313e66870 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:45.769688 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36247c81-dc69-4668-a536-812313e66870 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:45.769861 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36247c81-dc69-4668-a536-812313e66870 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:45.770108 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36247c81-dc69-4668-a536-812313e66870 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:45.788778 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36247c81-dc69-4668-a536-812313e66870 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:45.791035 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36247c81-dc69-4668-a536-812313e66870 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:45.802568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1077/2157] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:45 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:45.909408 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e3ab1-09e4-4070-8243-608e646ee924 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:45.920730 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e3ab1-09e4-4070-8243-608e646ee924 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:45.924922 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78425cff-f34b-4bd1-a324-2a145107778e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:45.926773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78425cff-f34b-4bd1-a324-2a145107778e service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:45.927167 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78425cff-f34b-4bd1-a324-2a145107778e service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:45.927450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78425cff-f34b-4bd1-a324-2a145107778e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:45.996790 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78425cff-f34b-4bd1-a324-2a145107778e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:45.998877 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78425cff-f34b-4bd1-a324-2a145107778e service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:46.009805 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1081/2158] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:46.557832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36247c81-dc69-4668-a536-812313e66870 service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:46.559316 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-36247c81-dc69-4668-a536-812313e66870 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:46.644030 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36247c81-dc69-4668-a536-812313e66870 service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:46.649279 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e8a6b-184d-4c83-9d04-3c2fcd5a3975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:46.649969 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e8a6b-184d-4c83-9d04-3c2fcd5a3975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:46.650344 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e8a6b-184d-4c83-9d04-3c2fcd5a3975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:46.650623 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e8a6b-184d-4c83-9d04-3c2fcd5a3975 admin admin] PATH_INFO: `/v3/users/e452beb1eaf540daa9bbfec92e3138cb` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:46.651915 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452e8a6b-184d-4c83-9d04-3c2fcd5a3975 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e452beb1eaf540daa9bbfec92e3138cb)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:46.654148 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452e8a6b-184d-4c83-9d04-3c2fcd5a3975 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:46.715972 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-452e8a6b-184d-4c83-9d04-3c2fcd5a3975 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e452beb1eaf540daa9bbfec92e3138cb'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:46.716703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-452e8a6b-184d-4c83-9d04-3c2fcd5a3975 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e452beb1eaf540daa9bbfec92e3138cb'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:46.732152 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-452e8a6b-184d-4c83-9d04-3c2fcd5a3975 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e452beb1eaf540daa9bbfec92e3138cb'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:46.744464 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-452e8a6b-184d-4c83-9d04-3c2fcd5a3975 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e452beb1eaf540daa9bbfec92e3138cb'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:46.750422 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:46.753116 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-452e8a6b-184d-4c83-9d04-3c2fcd5a3975 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e452beb1eaf540daa9bbfec92e3138cb'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:46.764935 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1078/2159] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:46 2022] DELETE /identity/v3/users/e452beb1eaf540daa9bbfec92e3138cb => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:46.971852 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78425cff-f34b-4bd1-a324-2a145107778e service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:46.977597 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:46.994771 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78425cff-f34b-4bd1-a324-2a145107778e service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:47.080573 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf0bc9-99ca-4b10-acf0-b5ccaf7fb9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:47.081751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf0bc9-99ca-4b10-acf0-b5ccaf7fb9b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:47.082154 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf0bc9-99ca-4b10-acf0-b5ccaf7fb9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:47.082526 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf0bc9-99ca-4b10-acf0-b5ccaf7fb9b1 admin admin] PATH_INFO: `/v3/projects/f35be82fde6249bab9008a41cd7ce8d9` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:47.084515 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf0bc9-99ca-4b10-acf0-b5ccaf7fb9b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f35be82fde6249bab9008a41cd7ce8d9)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:47.086764 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf0bc9-99ca-4b10-acf0-b5ccaf7fb9b1 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:47.141582 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-02cf0bc9-99ca-4b10-acf0-b5ccaf7fb9b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f35be82fde6249bab9008a41cd7ce8d9', 'request_id': 'req-02cf0bc9-99ca-4b10-acf0-b5ccaf7fb9b1'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:47.149480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-02cf0bc9-99ca-4b10-acf0-b5ccaf7fb9b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f35be82fde6249bab9008a41cd7ce8d9', 'request_id': 'req-02cf0bc9-99ca-4b10-acf0-b5ccaf7fb9b1'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:47.157143 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-02cf0bc9-99ca-4b10-acf0-b5ccaf7fb9b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f35be82fde6249bab9008a41cd7ce8d9', 'request_id': 'req-02cf0bc9-99ca-4b10-acf0-b5ccaf7fb9b1'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:47.170876 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-02cf0bc9-99ca-4b10-acf0-b5ccaf7fb9b1 admin admin] The token cache is being invalidate because project f35be82fde6249bab9008a41cd7ce8d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:47.171281 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-02cf0bc9-99ca-4b10-acf0-b5ccaf7fb9b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:47.172033 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-02cf0bc9-99ca-4b10-acf0-b5ccaf7fb9b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:47.173777 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1082/2160] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:46 2022] DELETE /identity/v3/projects/f35be82fde6249bab9008a41cd7ce8d9 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:47.179401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390eff5-0441-422b-bb3f-4fd977b2c21f None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:47.179694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390eff5-0441-422b-bb3f-4fd977b2c21f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:47.179977 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390eff5-0441-422b-bb3f-4fd977b2c21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:47.200467 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-2390eff5-0441-422b-bb3f-4fd977b2c21f None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:47.201837 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-2390eff5-0441-422b-bb3f-4fd977b2c21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:47.281876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1079/2161] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:47.289595 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e620ca-097e-40ad-8168-d70072bbba0a None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:47.289882 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e620ca-097e-40ad-8168-d70072bbba0a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:47.290213 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e620ca-097e-40ad-8168-d70072bbba0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:47.314692 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-11e620ca-097e-40ad-8168-d70072bbba0a None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:47.316732 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-11e620ca-097e-40ad-8168-d70072bbba0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:47.334391 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1083/2162] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:47.337045 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390eff5-0441-422b-bb3f-4fd977b2c21f None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:47.357577 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390eff5-0441-422b-bb3f-4fd977b2c21f None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:47.371593 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74184c02-f8db-48fa-9649-9e32e91f951c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:47.371593 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74184c02-f8db-48fa-9649-9e32e91f951c admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:47.371593 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74184c02-f8db-48fa-9649-9e32e91f951c admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:47.371593 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74184c02-f8db-48fa-9649-9e32e91f951c admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:47.372013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74184c02-f8db-48fa-9649-9e32e91f951c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:44:47.372013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74184c02-f8db-48fa-9649-9e32e91f951c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:47.372013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74184c02-f8db-48fa-9649-9e32e91f951c admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:47.388742 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1080/2163] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:44:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:44:47.393223 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e620ca-097e-40ad-8168-d70072bbba0a None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:47.405296 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e620ca-097e-40ad-8168-d70072bbba0a None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:47.411309 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c078102e-0331-4c07-b4f2-59052025b981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:47.412654 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078102e-0331-4c07-b4f2-59052025b981 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:47.412975 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078102e-0331-4c07-b4f2-59052025b981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:47.413259 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078102e-0331-4c07-b4f2-59052025b981 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:47.413964 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c078102e-0331-4c07-b4f2-59052025b981 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:47.416178 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c078102e-0331-4c07-b4f2-59052025b981 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:47.454816 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1084/2164] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:47 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:47.459535 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74184c02-f8db-48fa-9649-9e32e91f951c admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:47.473560 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74184c02-f8db-48fa-9649-9e32e91f951c admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:47.564294 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dfbb00-84b2-4234-9100-20c5a40fea39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:47.565238 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfbb00-84b2-4234-9100-20c5a40fea39 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:47.565751 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfbb00-84b2-4234-9100-20c5a40fea39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:47.566102 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfbb00-84b2-4234-9100-20c5a40fea39 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:47.566811 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dfbb00-84b2-4234-9100-20c5a40fea39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:47.568886 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dfbb00-84b2-4234-9100-20c5a40fea39 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:47.603932 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1081/2165] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:47 2022] POST /identity/v3/users => generated 288 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:47.608085 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c078102e-0331-4c07-b4f2-59052025b981 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:47.620020 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c078102e-0331-4c07-b4f2-59052025b981 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:47.625705 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33505ccf-85ea-456d-a23d-9bb3c139ac77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:47.626382 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33505ccf-85ea-456d-a23d-9bb3c139ac77 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:47.626674 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33505ccf-85ea-456d-a23d-9bb3c139ac77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:47.626951 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33505ccf-85ea-456d-a23d-9bb3c139ac77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:47.627764 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33505ccf-85ea-456d-a23d-9bb3c139ac77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:47.629498 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33505ccf-85ea-456d-a23d-9bb3c139ac77 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:47.643822 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1085/2166] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:47.648100 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dfbb00-84b2-4234-9100-20c5a40fea39 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:47.666281 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dfbb00-84b2-4234-9100-20c5a40fea39 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:47.673549 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb937cf-9ae9-4e00-be7c-72da0e366b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:47.674263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb937cf-9ae9-4e00-be7c-72da0e366b13 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:47.674638 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb937cf-9ae9-4e00-be7c-72da0e366b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:47.674916 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb937cf-9ae9-4e00-be7c-72da0e366b13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:47.675718 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb937cf-9ae9-4e00-be7c-72da0e366b13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:47.690845 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb937cf-9ae9-4e00-be7c-72da0e366b13 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:47.733573 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1082/2167] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:44:47.737594 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33505ccf-85ea-456d-a23d-9bb3c139ac77 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:47.763124 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33505ccf-85ea-456d-a23d-9bb3c139ac77 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:47.768734 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa6304-6bc8-4d89-b763-baf0ac7f0f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:47.769634 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa6304-6bc8-4d89-b763-baf0ac7f0f8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:47.769909 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa6304-6bc8-4d89-b763-baf0ac7f0f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:47.770195 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa6304-6bc8-4d89-b763-baf0ac7f0f8c admin admin] PATH_INFO: `/v3/projects/92c72b8fab7e4ac0a08f7207e9877013/users/936f1335f9894a9495c9c1598a6aaf1b/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:47.790457 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa6304-6bc8-4d89-b763-baf0ac7f0f8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=92c72b8fab7e4ac0a08f7207e9877013, user_id=936f1335f9894a9495c9c1598a6aaf1b, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:47.793542 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa6304-6bc8-4d89-b763-baf0ac7f0f8c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:47.813791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1086/2168] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:47 2022] PUT /identity/v3/projects/92c72b8fab7e4ac0a08f7207e9877013/users/936f1335f9894a9495c9c1598a6aaf1b/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:47.828759 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a985d1-525a-4b42-a7eb-4743d1a6bd93 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:47.828759 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a985d1-525a-4b42-a7eb-4743d1a6bd93 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:47.828759 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a985d1-525a-4b42-a7eb-4743d1a6bd93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:47.870853 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-89a985d1-525a-4b42-a7eb-4743d1a6bd93 None None] MFA Rules not processed for user `936f1335f9894a9495c9c1598a6aaf1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:47.872828 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-89a985d1-525a-4b42-a7eb-4743d1a6bd93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:47.942658 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:47.961226 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1083/2169] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:47.972652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa6304-6bc8-4d89-b763-baf0ac7f0f8c admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:47.973710 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6bfa6304-6bc8-4d89-b763-baf0ac7f0f8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:48.016742 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:48.054220 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa6304-6bc8-4d89-b763-baf0ac7f0f8c admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:48.059784 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195ffc9-50c2-4635-97ae-4da0222f8255 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:48.060446 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195ffc9-50c2-4635-97ae-4da0222f8255 service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:48.060728 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195ffc9-50c2-4635-97ae-4da0222f8255 service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:48.061004 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195ffc9-50c2-4635-97ae-4da0222f8255 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:48.070713 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195ffc9-50c2-4635-97ae-4da0222f8255 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:48.072377 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195ffc9-50c2-4635-97ae-4da0222f8255 service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:48.082461 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1087/2170] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:48.315185 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a985d1-525a-4b42-a7eb-4743d1a6bd93 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:48.394970 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a985d1-525a-4b42-a7eb-4743d1a6bd93 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:48.401127 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d79bc1-256b-4a7d-ab36-4d40b2d95eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:48.402078 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d79bc1-256b-4a7d-ab36-4d40b2d95eb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:48.402370 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d79bc1-256b-4a7d-ab36-4d40b2d95eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:48.402913 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d79bc1-256b-4a7d-ab36-4d40b2d95eb6 admin admin] PATH_INFO: `/v3/users/936f1335f9894a9495c9c1598a6aaf1b` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:48.404207 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d79bc1-256b-4a7d-ab36-4d40b2d95eb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=936f1335f9894a9495c9c1598a6aaf1b)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:48.406136 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d79bc1-256b-4a7d-ab36-4d40b2d95eb6 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:48.460889 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-00d79bc1-256b-4a7d-ab36-4d40b2d95eb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '936f1335f9894a9495c9c1598a6aaf1b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:48.461658 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-00d79bc1-256b-4a7d-ab36-4d40b2d95eb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '936f1335f9894a9495c9c1598a6aaf1b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:48.479170 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-00d79bc1-256b-4a7d-ab36-4d40b2d95eb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '936f1335f9894a9495c9c1598a6aaf1b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:48.488053 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-00d79bc1-256b-4a7d-ab36-4d40b2d95eb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '936f1335f9894a9495c9c1598a6aaf1b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:48.496285 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-00d79bc1-256b-4a7d-ab36-4d40b2d95eb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '936f1335f9894a9495c9c1598a6aaf1b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:48.503391 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1084/2171] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:48 2022] DELETE /identity/v3/users/936f1335f9894a9495c9c1598a6aaf1b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:48.509166 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f9faa-3dda-4d34-b91f-b09c34c6eb54 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:48.509701 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f9faa-3dda-4d34-b91f-b09c34c6eb54 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:48.510118 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f9faa-3dda-4d34-b91f-b09c34c6eb54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:48.531783 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-6f4f9faa-3dda-4d34-b91f-b09c34c6eb54 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:48.533325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6f4f9faa-3dda-4d34-b91f-b09c34c6eb54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:48.621219 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1088/2172] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:48.632352 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d79bc1-256b-4a7d-ab36-4d40b2d95eb6 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:48.634085 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-00d79bc1-256b-4a7d-ab36-4d40b2d95eb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:48.700699 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d79bc1-256b-4a7d-ab36-4d40b2d95eb6 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:48.705041 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d2c59-dde0-4981-a468-c5aefb62a04d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:48.705700 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d2c59-dde0-4981-a468-c5aefb62a04d service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:48.705978 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d2c59-dde0-4981-a468-c5aefb62a04d service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:48.706259 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d2c59-dde0-4981-a468-c5aefb62a04d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:48.717915 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6d2c59-dde0-4981-a468-c5aefb62a04d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:48.719341 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6d2c59-dde0-4981-a468-c5aefb62a04d service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:48.730755 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1085/2173] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:49.139562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f9faa-3dda-4d34-b91f-b09c34c6eb54 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:49.141138 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6f4f9faa-3dda-4d34-b91f-b09c34c6eb54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:49.154326 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f9faa-3dda-4d34-b91f-b09c34c6eb54 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:49.159414 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e6d987-47bf-4be6-be85-cace3d685306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:49.160573 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6d987-47bf-4be6-be85-cace3d685306 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:49.161043 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6d987-47bf-4be6-be85-cace3d685306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:49.162140 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6d987-47bf-4be6-be85-cace3d685306 admin admin] PATH_INFO: `/v3/projects/92c72b8fab7e4ac0a08f7207e9877013` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:49.163646 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e6d987-47bf-4be6-be85-cace3d685306 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92c72b8fab7e4ac0a08f7207e9877013)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:49.165868 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e6d987-47bf-4be6-be85-cace3d685306 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:49.231726 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d3e6d987-47bf-4be6-be85-cace3d685306 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92c72b8fab7e4ac0a08f7207e9877013', 'request_id': 'req-d3e6d987-47bf-4be6-be85-cace3d685306'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:49.245241 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d3e6d987-47bf-4be6-be85-cace3d685306 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92c72b8fab7e4ac0a08f7207e9877013', 'request_id': 'req-d3e6d987-47bf-4be6-be85-cace3d685306'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:49.255414 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d3e6d987-47bf-4be6-be85-cace3d685306 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92c72b8fab7e4ac0a08f7207e9877013', 'request_id': 'req-d3e6d987-47bf-4be6-be85-cace3d685306'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:49.267490 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d3e6d987-47bf-4be6-be85-cace3d685306 admin admin] The token cache is being invalidate because project 92c72b8fab7e4ac0a08f7207e9877013 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:49.267777 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d3e6d987-47bf-4be6-be85-cace3d685306 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:49.268785 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d3e6d987-47bf-4be6-be85-cace3d685306 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:49.270374 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1089/2174] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:49 2022] DELETE /identity/v3/projects/92c72b8fab7e4ac0a08f7207e9877013 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:49.277236 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a732e-8d35-4ce2-9483-1b3b4598f72f None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:49.277788 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a732e-8d35-4ce2-9483-1b3b4598f72f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:49.278070 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a732e-8d35-4ce2-9483-1b3b4598f72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:49.302618 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-225a732e-8d35-4ce2-9483-1b3b4598f72f None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:49.303968 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-225a732e-8d35-4ce2-9483-1b3b4598f72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:49.314446 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:49.380273 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1086/2175] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:49.388715 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3cd0ff-54ca-40ef-b61b-bfc8bc58f4d7 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:49.389006 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3cd0ff-54ca-40ef-b61b-bfc8bc58f4d7 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:49.389284 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3cd0ff-54ca-40ef-b61b-bfc8bc58f4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:49.408535 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-af3cd0ff-54ca-40ef-b61b-bfc8bc58f4d7 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:49.409827 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-af3cd0ff-54ca-40ef-b61b-bfc8bc58f4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:49.418812 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1090/2176] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:49.423302 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225a732e-8d35-4ce2-9483-1b3b4598f72f None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:49.433695 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225a732e-8d35-4ce2-9483-1b3b4598f72f None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:49.438367 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54169dda-5820-44ab-b8ff-32e3b2b6d9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:49.439059 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54169dda-5820-44ab-b8ff-32e3b2b6d9a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:49.439349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54169dda-5820-44ab-b8ff-32e3b2b6d9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:49.439625 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54169dda-5820-44ab-b8ff-32e3b2b6d9a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:49.440363 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54169dda-5820-44ab-b8ff-32e3b2b6d9a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:44:49.440673 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54169dda-5820-44ab-b8ff-32e3b2b6d9a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:49.442301 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54169dda-5820-44ab-b8ff-32e3b2b6d9a8 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:49.455534 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1087/2177] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:44:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:44:49.461703 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3cd0ff-54ca-40ef-b61b-bfc8bc58f4d7 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:49.471346 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3cd0ff-54ca-40ef-b61b-bfc8bc58f4d7 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:49.476010 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338eeab0-b610-4378-a062-e9314e677531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:49.476665 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338eeab0-b610-4378-a062-e9314e677531 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:49.476944 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338eeab0-b610-4378-a062-e9314e677531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:49.477209 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338eeab0-b610-4378-a062-e9314e677531 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:49.477887 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338eeab0-b610-4378-a062-e9314e677531 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:49.479881 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338eeab0-b610-4378-a062-e9314e677531 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:49.508823 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1091/2178] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:49 2022] POST /identity/v3/projects => generated 355 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:49.513298 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54169dda-5820-44ab-b8ff-32e3b2b6d9a8 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:49.524127 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54169dda-5820-44ab-b8ff-32e3b2b6d9a8 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:49.598842 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f062579-a4cb-42dc-a69d-8bd3eb899f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:49.599573 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f062579-a4cb-42dc-a69d-8bd3eb899f6d admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:49.599858 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f062579-a4cb-42dc-a69d-8bd3eb899f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:49.600133 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f062579-a4cb-42dc-a69d-8bd3eb899f6d admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:49.600829 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f062579-a4cb-42dc-a69d-8bd3eb899f6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:49.602757 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f062579-a4cb-42dc-a69d-8bd3eb899f6d admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:49.624410 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1088/2179] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:49 2022] POST /identity/v3/users => generated 278 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:49.628548 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338eeab0-b610-4378-a062-e9314e677531 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:49.639814 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338eeab0-b610-4378-a062-e9314e677531 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:49.644765 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2904c6-50da-46b2-9230-71cd4cac2696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:49.646330 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2904c6-50da-46b2-9230-71cd4cac2696 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:49.646330 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2904c6-50da-46b2-9230-71cd4cac2696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:49.646330 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2904c6-50da-46b2-9230-71cd4cac2696 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:49.646698 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2904c6-50da-46b2-9230-71cd4cac2696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:49.648415 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2904c6-50da-46b2-9230-71cd4cac2696 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:49.651693 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:49.662608 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1092/2180] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:49.667099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f062579-a4cb-42dc-a69d-8bd3eb899f6d admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:49.679304 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f062579-a4cb-42dc-a69d-8bd3eb899f6d admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:49.680762 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2904c6-50da-46b2-9230-71cd4cac2696 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:49.682610 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-2f2904c6-50da-46b2-9230-71cd4cac2696 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:49.685417 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07e1864-8598-4e6a-81c7-99b2f6a53cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:49.686123 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e1864-8598-4e6a-81c7-99b2f6a53cb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:49.686414 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e1864-8598-4e6a-81c7-99b2f6a53cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:49.686695 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e1864-8598-4e6a-81c7-99b2f6a53cb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:49.687473 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07e1864-8598-4e6a-81c7-99b2f6a53cb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:49.689087 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07e1864-8598-4e6a-81c7-99b2f6a53cb2 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:49.702142 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1089/2181] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:49.704453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2904c6-50da-46b2-9230-71cd4cac2696 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:49.706767 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07e1864-8598-4e6a-81c7-99b2f6a53cb2 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:49.716770 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39349a39-03b9-46ff-959a-f8c088139211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:49.718124 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39349a39-03b9-46ff-959a-f8c088139211 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:49.718540 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39349a39-03b9-46ff-959a-f8c088139211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:49.719089 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39349a39-03b9-46ff-959a-f8c088139211 admin admin] PATH_INFO: `/v3/users/9d29a989cf0848b4936b059b0c281098` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:49.721309 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39349a39-03b9-46ff-959a-f8c088139211 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d29a989cf0848b4936b059b0c281098)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:49.721461 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07e1864-8598-4e6a-81c7-99b2f6a53cb2 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:49.723968 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39349a39-03b9-46ff-959a-f8c088139211 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:49.726320 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8791af-6b08-490d-805b-d326d8879c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:49.727251 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8791af-6b08-490d-805b-d326d8879c4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:49.727540 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8791af-6b08-490d-805b-d326d8879c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:49.727825 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8791af-6b08-490d-805b-d326d8879c4c admin admin] PATH_INFO: `/v3/projects/4b8b91c41d98497a849e69acf55912a8/users/6ff94fb079d24ddc882d8f8be276dbb4/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:49.744845 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8791af-6b08-490d-805b-d326d8879c4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b8b91c41d98497a849e69acf55912a8, user_id=6ff94fb079d24ddc882d8f8be276dbb4, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:49.747546 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8791af-6b08-490d-805b-d326d8879c4c admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:49.764969 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1090/2182] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:49 2022] PUT /identity/v3/projects/4b8b91c41d98497a849e69acf55912a8/users/6ff94fb079d24ddc882d8f8be276dbb4/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:49.776945 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fdc04-4246-40a4-a1ba-d4abc5f5d8ba None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:49.777232 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fdc04-4246-40a4-a1ba-d4abc5f5d8ba None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:49.777768 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fdc04-4246-40a4-a1ba-d4abc5f5d8ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:49.806231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-39349a39-03b9-46ff-959a-f8c088139211 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d29a989cf0848b4936b059b0c281098'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:49.806944 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-39349a39-03b9-46ff-959a-f8c088139211 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d29a989cf0848b4936b059b0c281098'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:49.811372 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-597fdc04-4246-40a4-a1ba-d4abc5f5d8ba None None] MFA Rules not processed for user `6ff94fb079d24ddc882d8f8be276dbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:49.812581 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-597fdc04-4246-40a4-a1ba-d4abc5f5d8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:49.822624 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-39349a39-03b9-46ff-959a-f8c088139211 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d29a989cf0848b4936b059b0c281098'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:49.823427 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-39349a39-03b9-46ff-959a-f8c088139211 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d29a989cf0848b4936b059b0c281098'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:49.833110 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-39349a39-03b9-46ff-959a-f8c088139211 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d29a989cf0848b4936b059b0c281098'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:49.848231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1093/2183] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:49 2022] DELETE /identity/v3/users/9d29a989cf0848b4936b059b0c281098 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:49.853465 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03820a-fbbb-4543-a7e5-eee1380de0b0 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:49.853639 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03820a-fbbb-4543-a7e5-eee1380de0b0 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:49.853899 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03820a-fbbb-4543-a7e5-eee1380de0b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:49.872709 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-aa03820a-fbbb-4543-a7e5-eee1380de0b0 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:49.872917 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:49.873971 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-aa03820a-fbbb-4543-a7e5-eee1380de0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:49.905644 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1091/2184] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:49 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:44:49.910314 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597fdc04-4246-40a4-a1ba-d4abc5f5d8ba None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:49.912814 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-597fdc04-4246-40a4-a1ba-d4abc5f5d8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:49.941015 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1094/2185] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:44:49.955017 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03820a-fbbb-4543-a7e5-eee1380de0b0 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:49.957764 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-aa03820a-fbbb-4543-a7e5-eee1380de0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:50.003386 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597fdc04-4246-40a4-a1ba-d4abc5f5d8ba None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:50.008272 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507be50-1afe-45f5-94ae-07baa9455288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:50.008926 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507be50-1afe-45f5-94ae-07baa9455288 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:50.009212 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507be50-1afe-45f5-94ae-07baa9455288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:50.009494 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507be50-1afe-45f5-94ae-07baa9455288 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:50.010222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2507be50-1afe-45f5-94ae-07baa9455288 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:50.012173 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2507be50-1afe-45f5-94ae-07baa9455288 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:50.036370 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03820a-fbbb-4543-a7e5-eee1380de0b0 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:50.041277 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c796c-e0d7-439d-9a98-36e9d2739f81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:50.041561 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1092/2186] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:49 2022] POST /identity/v3/projects => generated 353 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:44:50.043776 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c796c-e0d7-439d-9a98-36e9d2739f81 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:50.044125 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c796c-e0d7-439d-9a98-36e9d2739f81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:50.044605 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c796c-e0d7-439d-9a98-36e9d2739f81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:50.046256 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507be50-1afe-45f5-94ae-07baa9455288 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:50.058987 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123c796c-e0d7-439d-9a98-36e9d2739f81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:50.060792 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123c796c-e0d7-439d-9a98-36e9d2739f81 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:50.072472 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1095/2187] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:44:50.129797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507be50-1afe-45f5-94ae-07baa9455288 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:50.136360 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd3dd2-f866-4224-b368-8be973fd03cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:50.137092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd3dd2-f866-4224-b368-8be973fd03cc admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:50.137402 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd3dd2-f866-4224-b368-8be973fd03cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:50.137653 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd3dd2-f866-4224-b368-8be973fd03cc admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:50.138609 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bd3dd2-f866-4224-b368-8be973fd03cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:50.140617 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bd3dd2-f866-4224-b368-8be973fd03cc admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:50.171391 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1093/2188] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:50 2022] POST /identity/v3/users => generated 283 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:44:50.175263 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c796c-e0d7-439d-9a98-36e9d2739f81 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:50.185616 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c796c-e0d7-439d-9a98-36e9d2739f81 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:50.195491 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0dd1d-63cb-4e06-bdd1-c5c37e4ea193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:50.196186 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0dd1d-63cb-4e06-bdd1-c5c37e4ea193 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:50.196477 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0dd1d-63cb-4e06-bdd1-c5c37e4ea193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:50.196759 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0dd1d-63cb-4e06-bdd1-c5c37e4ea193 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:50.197514 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f0dd1d-63cb-4e06-bdd1-c5c37e4ea193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:50.199213 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f0dd1d-63cb-4e06-bdd1-c5c37e4ea193 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:50.212109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1096/2189] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:50.217105 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd3dd2-f866-4224-b368-8be973fd03cc admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:50.232871 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd3dd2-f866-4224-b368-8be973fd03cc admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:50.240405 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db54e2f-2b93-4056-9602-46edd8a72187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:50.241318 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db54e2f-2b93-4056-9602-46edd8a72187 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:50.241610 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db54e2f-2b93-4056-9602-46edd8a72187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:50.241885 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db54e2f-2b93-4056-9602-46edd8a72187 admin admin] PATH_INFO: `/v3/projects/0b55d6495ef849ffb8e41a202537feb1/users/cbe5f30386f24344822043be00429b27/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:50.262634 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db54e2f-2b93-4056-9602-46edd8a72187 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b55d6495ef849ffb8e41a202537feb1, user_id=cbe5f30386f24344822043be00429b27, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:50.265614 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db54e2f-2b93-4056-9602-46edd8a72187 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:50.285185 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1094/2190] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:50 2022] PUT /identity/v3/projects/0b55d6495ef849ffb8e41a202537feb1/users/cbe5f30386f24344822043be00429b27/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:50.290321 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0dd1d-63cb-4e06-bdd1-c5c37e4ea193 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:50.296035 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:50.304599 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0dd1d-63cb-4e06-bdd1-c5c37e4ea193 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:50.405102 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efcf335-de6c-4325-b39f-416779198aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:50.405792 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcf335-de6c-4325-b39f-416779198aca admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:50.406079 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcf335-de6c-4325-b39f-416779198aca admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:50.406356 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcf335-de6c-4325-b39f-416779198aca admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:50.407275 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efcf335-de6c-4325-b39f-416779198aca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:50.409019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efcf335-de6c-4325-b39f-416779198aca admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:50.424279 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1097/2191] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:50 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:50.425414 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db54e2f-2b93-4056-9602-46edd8a72187 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:50.436792 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db54e2f-2b93-4056-9602-46edd8a72187 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:50.441864 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb42b5f-c07f-4b31-bfb9-8c1fe73b2000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:50.443551 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb42b5f-c07f-4b31-bfb9-8c1fe73b2000 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:50.444059 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb42b5f-c07f-4b31-bfb9-8c1fe73b2000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:50.444353 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb42b5f-c07f-4b31-bfb9-8c1fe73b2000 admin admin] PATH_INFO: `/v3/projects/0b55d6495ef849ffb8e41a202537feb1/users/cbe5f30386f24344822043be00429b27/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:50.446390 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb42b5f-c07f-4b31-bfb9-8c1fe73b2000 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b55d6495ef849ffb8e41a202537feb1, user_id=cbe5f30386f24344822043be00429b27, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:50.449377 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb42b5f-c07f-4b31-bfb9-8c1fe73b2000 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:50.461883 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1095/2192] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:50 2022] PUT /identity/v3/projects/0b55d6495ef849ffb8e41a202537feb1/users/cbe5f30386f24344822043be00429b27/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:50.472558 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efcf335-de6c-4325-b39f-416779198aca admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:50.473300 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fcd55a-b565-4719-a674-ff429e1e6ad9 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:50.473596 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fcd55a-b565-4719-a674-ff429e1e6ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:50.473879 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fcd55a-b565-4719-a674-ff429e1e6ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:50.475519 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6efcf335-de6c-4325-b39f-416779198aca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:50.493362 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efcf335-de6c-4325-b39f-416779198aca admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:50.527874 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-d4fcd55a-b565-4719-a674-ff429e1e6ad9 None None] MFA Rules not processed for user `cbe5f30386f24344822043be00429b27`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:50.529120 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d4fcd55a-b565-4719-a674-ff429e1e6ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:50.545624 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:50.573899 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac5ee02-575e-49a0-8230-6a7bbc67f6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:50.574773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5ee02-575e-49a0-8230-6a7bbc67f6e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:50.575091 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5ee02-575e-49a0-8230-6a7bbc67f6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:50.575378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5ee02-575e-49a0-8230-6a7bbc67f6e5 admin admin] PATH_INFO: `/v3/projects/1c49db0fb633401c85b2875600a04408` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:50.576592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac5ee02-575e-49a0-8230-6a7bbc67f6e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c49db0fb633401c85b2875600a04408)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:50.578519 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac5ee02-575e-49a0-8230-6a7bbc67f6e5 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:50.608970 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1096/2193] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:50 2022] POST /identity/v3/auth/tokens => generated 3287 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:44:50.620813 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fcd55a-b565-4719-a674-ff429e1e6ad9 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:50.622210 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d4fcd55a-b565-4719-a674-ff429e1e6ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:50.627639 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-bac5ee02-575e-49a0-8230-6a7bbc67f6e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c49db0fb633401c85b2875600a04408', 'request_id': 'req-bac5ee02-575e-49a0-8230-6a7bbc67f6e5'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:50.637444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-bac5ee02-575e-49a0-8230-6a7bbc67f6e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c49db0fb633401c85b2875600a04408', 'request_id': 'req-bac5ee02-575e-49a0-8230-6a7bbc67f6e5'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:50.643939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-bac5ee02-575e-49a0-8230-6a7bbc67f6e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c49db0fb633401c85b2875600a04408', 'request_id': 'req-bac5ee02-575e-49a0-8230-6a7bbc67f6e5'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:50.655756 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-bac5ee02-575e-49a0-8230-6a7bbc67f6e5 admin admin] The token cache is being invalidate because project 1c49db0fb633401c85b2875600a04408 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:50.656047 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-bac5ee02-575e-49a0-8230-6a7bbc67f6e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:50.656770 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-bac5ee02-575e-49a0-8230-6a7bbc67f6e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:50.658379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1098/2194] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:50 2022] DELETE /identity/v3/projects/1c49db0fb633401c85b2875600a04408 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:50.662225 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac5ee02-575e-49a0-8230-6a7bbc67f6e5 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:50.663612 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-bac5ee02-575e-49a0-8230-6a7bbc67f6e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:50.680902 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fcd55a-b565-4719-a674-ff429e1e6ad9 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:50.695256 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:50.733958 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac5ee02-575e-49a0-8230-6a7bbc67f6e5 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:50.738621 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1923b97c-b94f-436e-b15a-8c81dbd7a8eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:50.738760 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ecb55c-08a4-4aae-b073-729f0feb9556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:50.739314 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923b97c-b94f-436e-b15a-8c81dbd7a8eb service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:50.739640 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923b97c-b94f-436e-b15a-8c81dbd7a8eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:50.739738 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ecb55c-08a4-4aae-b073-729f0feb9556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:50.739923 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923b97c-b94f-436e-b15a-8c81dbd7a8eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:50.740013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ecb55c-08a4-4aae-b073-729f0feb9556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:50.740360 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ecb55c-08a4-4aae-b073-729f0feb9556 admin admin] PATH_INFO: `/v3/users/32b777358749425da8cea0e9804ddc3d` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:50.741581 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ecb55c-08a4-4aae-b073-729f0feb9556 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32b777358749425da8cea0e9804ddc3d)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:50.741717 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1923b97c-b94f-436e-b15a-8c81dbd7a8eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:50.743553 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ecb55c-08a4-4aae-b073-729f0feb9556 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:50.794308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f4ecb55c-08a4-4aae-b073-729f0feb9556 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32b777358749425da8cea0e9804ddc3d'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:50.795148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f4ecb55c-08a4-4aae-b073-729f0feb9556 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32b777358749425da8cea0e9804ddc3d'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:50.808438 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f4ecb55c-08a4-4aae-b073-729f0feb9556 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32b777358749425da8cea0e9804ddc3d'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:50.809149 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f4ecb55c-08a4-4aae-b073-729f0feb9556 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32b777358749425da8cea0e9804ddc3d'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:50.809551 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1923b97c-b94f-436e-b15a-8c81dbd7a8eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:50.811242 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1923b97c-b94f-436e-b15a-8c81dbd7a8eb service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:50.819117 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f4ecb55c-08a4-4aae-b073-729f0feb9556 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32b777358749425da8cea0e9804ddc3d'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:50.823775 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1097/2195] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:50 2022] GET /identity/v3/auth/tokens => generated 3287 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:44:50.834703 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1099/2196] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:50 2022] DELETE /identity/v3/users/32b777358749425da8cea0e9804ddc3d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:50.905843 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1923b97c-b94f-436e-b15a-8c81dbd7a8eb service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:50.907523 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1923b97c-b94f-436e-b15a-8c81dbd7a8eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:50.920454 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1923b97c-b94f-436e-b15a-8c81dbd7a8eb service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:50.932824 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:50.977781 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb539f-9021-40cc-b4f0-22dc87ea01ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:50.978547 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb539f-9021-40cc-b4f0-22dc87ea01ff service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:50.978837 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb539f-9021-40cc-b4f0-22dc87ea01ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:50.979239 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb539f-9021-40cc-b4f0-22dc87ea01ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:50.981077 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e3eb539f-9021-40cc-b4f0-22dc87ea01ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:51.050875 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb539f-9021-40cc-b4f0-22dc87ea01ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:51.052412 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb539f-9021-40cc-b4f0-22dc87ea01ff service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:51.061458 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1098/2197] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:50 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:51.128860 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ecb55c-08a4-4aae-b073-729f0feb9556 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:51.130363 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f4ecb55c-08a4-4aae-b073-729f0feb9556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:51.217472 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ecb55c-08a4-4aae-b073-729f0feb9556 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:51.222101 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18b4006-c567-4a4b-a6b5-9fbaeaf88664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:51.222959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b4006-c567-4a4b-a6b5-9fbaeaf88664 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:51.223278 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b4006-c567-4a4b-a6b5-9fbaeaf88664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:51.223553 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b4006-c567-4a4b-a6b5-9fbaeaf88664 admin admin] PATH_INFO: `/v3/projects/ad5c2864ddfe424e8ec446ee69fab016` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:51.224807 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18b4006-c567-4a4b-a6b5-9fbaeaf88664 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad5c2864ddfe424e8ec446ee69fab016)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:51.226784 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18b4006-c567-4a4b-a6b5-9fbaeaf88664 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:51.273988 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d18b4006-c567-4a4b-a6b5-9fbaeaf88664 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad5c2864ddfe424e8ec446ee69fab016', 'request_id': 'req-d18b4006-c567-4a4b-a6b5-9fbaeaf88664'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:51.275665 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:51.282586 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d18b4006-c567-4a4b-a6b5-9fbaeaf88664 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad5c2864ddfe424e8ec446ee69fab016', 'request_id': 'req-d18b4006-c567-4a4b-a6b5-9fbaeaf88664'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:51.289986 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d18b4006-c567-4a4b-a6b5-9fbaeaf88664 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad5c2864ddfe424e8ec446ee69fab016', 'request_id': 'req-d18b4006-c567-4a4b-a6b5-9fbaeaf88664'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:51.301508 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d18b4006-c567-4a4b-a6b5-9fbaeaf88664 admin admin] The token cache is being invalidate because project ad5c2864ddfe424e8ec446ee69fab016 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:51.301787 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d18b4006-c567-4a4b-a6b5-9fbaeaf88664 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:51.302474 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d18b4006-c567-4a4b-a6b5-9fbaeaf88664 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:51.304069 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1100/2198] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:51 2022] DELETE /identity/v3/projects/ad5c2864ddfe424e8ec446ee69fab016 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:51.309935 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59d4bd-c460-4d67-a97e-b30420efb30f None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:51.310310 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59d4bd-c460-4d67-a97e-b30420efb30f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:51.310677 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59d4bd-c460-4d67-a97e-b30420efb30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:51.333996 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-9f59d4bd-c460-4d67-a97e-b30420efb30f None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:51.335460 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-9f59d4bd-c460-4d67-a97e-b30420efb30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:51.399791 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1099/2199] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:51.407969 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4450d6-d169-4fa5-8594-1ff821da543c None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:51.408255 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4450d6-d169-4fa5-8594-1ff821da543c None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:51.408529 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4450d6-d169-4fa5-8594-1ff821da543c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:51.428569 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-bb4450d6-d169-4fa5-8594-1ff821da543c None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:51.429804 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-bb4450d6-d169-4fa5-8594-1ff821da543c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:51.443063 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1101/2200] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:51.447373 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f59d4bd-c460-4d67-a97e-b30420efb30f None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:51.457683 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f59d4bd-c460-4d67-a97e-b30420efb30f None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:51.463012 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f8855-e65d-49ef-a603-baade16a497e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:51.463713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f8855-e65d-49ef-a603-baade16a497e admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:51.464000 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f8855-e65d-49ef-a603-baade16a497e admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:51.464279 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f8855-e65d-49ef-a603-baade16a497e admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:51.465020 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f8855-e65d-49ef-a603-baade16a497e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:44:51.465328 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f8855-e65d-49ef-a603-baade16a497e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:51.467815 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f8855-e65d-49ef-a603-baade16a497e admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:51.481390 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1100/2201] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:44:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:44:51.486163 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4450d6-d169-4fa5-8594-1ff821da543c None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:51.496869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4450d6-d169-4fa5-8594-1ff821da543c None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:51.501906 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15bc7f-3788-4d81-b2ad-8142fb53037e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:51.502596 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15bc7f-3788-4d81-b2ad-8142fb53037e admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:51.502896 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15bc7f-3788-4d81-b2ad-8142fb53037e admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:51.503214 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15bc7f-3788-4d81-b2ad-8142fb53037e admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:51.503919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15bc7f-3788-4d81-b2ad-8142fb53037e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:51.506398 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15bc7f-3788-4d81-b2ad-8142fb53037e admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:51.541391 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1102/2202] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:51 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:51.544874 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f8855-e65d-49ef-a603-baade16a497e admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:51.556031 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f8855-e65d-49ef-a603-baade16a497e admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:51.632404 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:51.644591 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f9145-0cc5-458b-af1c-9e7aace2dd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:51.645437 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f9145-0cc5-458b-af1c-9e7aace2dd2d admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:51.645614 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f9145-0cc5-458b-af1c-9e7aace2dd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:51.646299 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f9145-0cc5-458b-af1c-9e7aace2dd2d admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:51.647048 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f9145-0cc5-458b-af1c-9e7aace2dd2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:51.649148 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f9145-0cc5-458b-af1c-9e7aace2dd2d admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:51.676353 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1101/2203] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:51 2022] POST /identity/v3/users => generated 292 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:44:51.682005 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15bc7f-3788-4d81-b2ad-8142fb53037e admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:51.695398 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15bc7f-3788-4d81-b2ad-8142fb53037e admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:51.700444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4295faf-3618-46fe-839d-2e84342ad6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:51.701117 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4295faf-3618-46fe-839d-2e84342ad6d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:51.701398 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4295faf-3618-46fe-839d-2e84342ad6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:51.701681 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4295faf-3618-46fe-839d-2e84342ad6d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:51.702423 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4295faf-3618-46fe-839d-2e84342ad6d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:51.704084 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4295faf-3618-46fe-839d-2e84342ad6d9 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:51.718522 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1103/2204] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:51.723841 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f9145-0cc5-458b-af1c-9e7aace2dd2d admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:51.735757 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f9145-0cc5-458b-af1c-9e7aace2dd2d admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:51.741123 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d774f-a27f-4936-bf47-3eeb4435f4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:51.741792 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d774f-a27f-4936-bf47-3eeb4435f4d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:51.742083 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d774f-a27f-4936-bf47-3eeb4435f4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:51.742363 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d774f-a27f-4936-bf47-3eeb4435f4d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:51.743120 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9d774f-a27f-4936-bf47-3eeb4435f4d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:51.744766 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9d774f-a27f-4936-bf47-3eeb4435f4d4 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:51.756118 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1102/2205] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:51.760591 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4295faf-3618-46fe-839d-2e84342ad6d9 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:51.770815 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4295faf-3618-46fe-839d-2e84342ad6d9 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:51.775627 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779862e-044c-487f-8137-7dd2ce8312e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:51.776567 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779862e-044c-487f-8137-7dd2ce8312e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:51.776844 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779862e-044c-487f-8137-7dd2ce8312e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:51.777113 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779862e-044c-487f-8137-7dd2ce8312e9 admin admin] PATH_INFO: `/v3/projects/54c81bbc5c054282975e372448830fe1/users/81683401473241bb94ac4ffdc96435d4/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:51.791874 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e779862e-044c-487f-8137-7dd2ce8312e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54c81bbc5c054282975e372448830fe1, user_id=81683401473241bb94ac4ffdc96435d4, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:51.794718 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e779862e-044c-487f-8137-7dd2ce8312e9 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:51.815067 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1104/2206] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:51 2022] PUT /identity/v3/projects/54c81bbc5c054282975e372448830fe1/users/81683401473241bb94ac4ffdc96435d4/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:51.823386 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4b777-b25d-46ac-81ad-5b69902da476 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:51.823676 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4b777-b25d-46ac-81ad-5b69902da476 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:51.823956 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4b777-b25d-46ac-81ad-5b69902da476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:51.850322 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-e9f4b777-b25d-46ac-81ad-5b69902da476 None None] MFA Rules not processed for user `81683401473241bb94ac4ffdc96435d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:51.851582 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e9f4b777-b25d-46ac-81ad-5b69902da476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:51.922484 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1103/2207] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:51.934286 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779862e-044c-487f-8137-7dd2ce8312e9 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:51.936071 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e779862e-044c-487f-8137-7dd2ce8312e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:51.992370 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779862e-044c-487f-8137-7dd2ce8312e9 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:51.996058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792d7d5-2d30-4a7a-a643-2d2087afbb82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:51.996739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792d7d5-2d30-4a7a-a643-2d2087afbb82 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:51.997023 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792d7d5-2d30-4a7a-a643-2d2087afbb82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:51.997299 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792d7d5-2d30-4a7a-a643-2d2087afbb82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:52.005598 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9792d7d5-2d30-4a7a-a643-2d2087afbb82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:52.007055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9792d7d5-2d30-4a7a-a643-2d2087afbb82 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:52.016103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1105/2208] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:55.614526 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f4b777-b25d-46ac-81ad-5b69902da476 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:55.688309 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f4b777-b25d-46ac-81ad-5b69902da476 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:55.693385 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6c7d3-4802-4812-84b4-2e0fe6026160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:55.694253 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6c7d3-4802-4812-84b4-2e0fe6026160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:55.694535 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6c7d3-4802-4812-84b4-2e0fe6026160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:55.694808 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6c7d3-4802-4812-84b4-2e0fe6026160 admin admin] PATH_INFO: `/v3/users/81683401473241bb94ac4ffdc96435d4` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:55.696029 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc6c7d3-4802-4812-84b4-2e0fe6026160 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81683401473241bb94ac4ffdc96435d4)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:55.697945 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc6c7d3-4802-4812-84b4-2e0fe6026160 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:55.754257 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0cc6c7d3-4802-4812-84b4-2e0fe6026160 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81683401473241bb94ac4ffdc96435d4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:55.754908 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0cc6c7d3-4802-4812-84b4-2e0fe6026160 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81683401473241bb94ac4ffdc96435d4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:55.767632 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0cc6c7d3-4802-4812-84b4-2e0fe6026160 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81683401473241bb94ac4ffdc96435d4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:55.769309 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:55.780404 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0cc6c7d3-4802-4812-84b4-2e0fe6026160 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81683401473241bb94ac4ffdc96435d4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:55.791722 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0cc6c7d3-4802-4812-84b4-2e0fe6026160 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81683401473241bb94ac4ffdc96435d4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:55.798372 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1104/2209] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:55 2022] DELETE /identity/v3/users/81683401473241bb94ac4ffdc96435d4 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:55.802825 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2dc3a2-a346-47f3-99fa-a3e9a44c6cb7 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:55.803354 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2dc3a2-a346-47f3-99fa-a3e9a44c6cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:55.803561 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2dc3a2-a346-47f3-99fa-a3e9a44c6cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:55.821490 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-1b2dc3a2-a346-47f3-99fa-a3e9a44c6cb7 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:55.822695 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1b2dc3a2-a346-47f3-99fa-a3e9a44c6cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:55.892104 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1106/2210] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:55.905287 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6c7d3-4802-4812-84b4-2e0fe6026160 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:55.908080 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-0cc6c7d3-4802-4812-84b4-2e0fe6026160 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:55.976298 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6c7d3-4802-4812-84b4-2e0fe6026160 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:55.980976 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fee9e4-937f-4078-b87f-63536aa0ca9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:55.981672 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fee9e4-937f-4078-b87f-63536aa0ca9f service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:55.982080 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fee9e4-937f-4078-b87f-63536aa0ca9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:55.982349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fee9e4-937f-4078-b87f-63536aa0ca9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:55.992253 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fee9e4-937f-4078-b87f-63536aa0ca9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:55.993686 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fee9e4-937f-4078-b87f-63536aa0ca9f service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:56.003659 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1105/2211] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:56.298333 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2dc3a2-a346-47f3-99fa-a3e9a44c6cb7 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:56.299804 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1b2dc3a2-a346-47f3-99fa-a3e9a44c6cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:56.312945 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2dc3a2-a346-47f3-99fa-a3e9a44c6cb7 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:56.318456 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bef27d-9ddf-47d8-ab60-a82054048f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:56.319383 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bef27d-9ddf-47d8-ab60-a82054048f18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:56.319670 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bef27d-9ddf-47d8-ab60-a82054048f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:56.320103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bef27d-9ddf-47d8-ab60-a82054048f18 admin admin] PATH_INFO: `/v3/projects/54c81bbc5c054282975e372448830fe1` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:56.321233 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bef27d-9ddf-47d8-ab60-a82054048f18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54c81bbc5c054282975e372448830fe1)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:56.323229 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bef27d-9ddf-47d8-ab60-a82054048f18 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:56.336083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:56.375837 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e7bef27d-9ddf-47d8-ab60-a82054048f18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54c81bbc5c054282975e372448830fe1', 'request_id': 'req-e7bef27d-9ddf-47d8-ab60-a82054048f18'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:56.386608 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e7bef27d-9ddf-47d8-ab60-a82054048f18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54c81bbc5c054282975e372448830fe1', 'request_id': 'req-e7bef27d-9ddf-47d8-ab60-a82054048f18'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:56.393897 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e7bef27d-9ddf-47d8-ab60-a82054048f18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54c81bbc5c054282975e372448830fe1', 'request_id': 'req-e7bef27d-9ddf-47d8-ab60-a82054048f18'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:56.405576 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e7bef27d-9ddf-47d8-ab60-a82054048f18 admin admin] The token cache is being invalidate because project 54c81bbc5c054282975e372448830fe1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:56.405909 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e7bef27d-9ddf-47d8-ab60-a82054048f18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:56.406510 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e7bef27d-9ddf-47d8-ab60-a82054048f18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:56.408820 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1107/2212] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:56 2022] DELETE /identity/v3/projects/54c81bbc5c054282975e372448830fe1 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:56.415953 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b3456f-2464-4d2d-b53f-f9a24a465c84 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:56.416285 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b3456f-2464-4d2d-b53f-f9a24a465c84 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:56.416531 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b3456f-2464-4d2d-b53f-f9a24a465c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:56.449869 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-28b3456f-2464-4d2d-b53f-f9a24a465c84 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:56.451108 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-28b3456f-2464-4d2d-b53f-f9a24a465c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:56.510014 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1106/2213] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:56.518788 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23e23e-ea74-4944-a0b8-b0935b410492 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:56.519102 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23e23e-ea74-4944-a0b8-b0935b410492 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:56.519384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23e23e-ea74-4944-a0b8-b0935b410492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:56.538647 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-1e23e23e-ea74-4944-a0b8-b0935b410492 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:56.540014 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1e23e23e-ea74-4944-a0b8-b0935b410492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:56.548214 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1108/2214] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:56.553041 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b3456f-2464-4d2d-b53f-f9a24a465c84 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:56.563111 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b3456f-2464-4d2d-b53f-f9a24a465c84 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:56.567976 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b5462b-78cc-4810-94f0-583d98afce9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:56.568630 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5462b-78cc-4810-94f0-583d98afce9d admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:56.568916 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5462b-78cc-4810-94f0-583d98afce9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:56.569197 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5462b-78cc-4810-94f0-583d98afce9d admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:56.569920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b5462b-78cc-4810-94f0-583d98afce9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:44:56.570230 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b5462b-78cc-4810-94f0-583d98afce9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:56.571886 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b5462b-78cc-4810-94f0-583d98afce9d admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:56.585977 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1107/2215] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:44:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:44:56.590032 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e23e23e-ea74-4944-a0b8-b0935b410492 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:56.601724 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e23e23e-ea74-4944-a0b8-b0935b410492 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:56.607135 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f213cdbb-3b94-4d92-9097-056e4c9519cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:56.607888 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f213cdbb-3b94-4d92-9097-056e4c9519cb admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:56.608206 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f213cdbb-3b94-4d92-9097-056e4c9519cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:56.608560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f213cdbb-3b94-4d92-9097-056e4c9519cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:56.609291 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f213cdbb-3b94-4d92-9097-056e4c9519cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:56.611475 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f213cdbb-3b94-4d92-9097-056e4c9519cb admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:56.639845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1109/2216] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:56 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:56.644030 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b5462b-78cc-4810-94f0-583d98afce9d admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:56.655759 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b5462b-78cc-4810-94f0-583d98afce9d admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:56.750407 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf22c8-c975-4fa7-b670-68d4e9c9ffab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:56.751301 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf22c8-c975-4fa7-b670-68d4e9c9ffab admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:56.751676 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf22c8-c975-4fa7-b670-68d4e9c9ffab admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:56.752034 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf22c8-c975-4fa7-b670-68d4e9c9ffab admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:56.754413 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cf22c8-c975-4fa7-b670-68d4e9c9ffab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:56.755037 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cf22c8-c975-4fa7-b670-68d4e9c9ffab admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:56.767655 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:56.792670 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1108/2217] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:56 2022] POST /identity/v3/users => generated 286 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:56.799477 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f213cdbb-3b94-4d92-9097-056e4c9519cb admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:56.813377 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f213cdbb-3b94-4d92-9097-056e4c9519cb admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:56.819495 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c5e38-e810-4e6d-9bb9-8c2b4cc99922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:56.820285 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c5e38-e810-4e6d-9bb9-8c2b4cc99922 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:56.820563 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c5e38-e810-4e6d-9bb9-8c2b4cc99922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:56.820842 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c5e38-e810-4e6d-9bb9-8c2b4cc99922 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:56.821591 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c5e38-e810-4e6d-9bb9-8c2b4cc99922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:56.824188 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c5e38-e810-4e6d-9bb9-8c2b4cc99922 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:56.842617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1110/2218] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:56.844305 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf22c8-c975-4fa7-b670-68d4e9c9ffab admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:56.857814 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf22c8-c975-4fa7-b670-68d4e9c9ffab admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:56.864481 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca09db-2846-4a4e-8fe6-ac3bed668398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:56.865454 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca09db-2846-4a4e-8fe6-ac3bed668398 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:56.867316 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca09db-2846-4a4e-8fe6-ac3bed668398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:56.868414 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca09db-2846-4a4e-8fe6-ac3bed668398 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:56.869459 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aca09db-2846-4a4e-8fe6-ac3bed668398 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:56.871281 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aca09db-2846-4a4e-8fe6-ac3bed668398 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:56.883408 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1109/2219] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:56.887579 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c5e38-e810-4e6d-9bb9-8c2b4cc99922 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:56.902128 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c5e38-e810-4e6d-9bb9-8c2b4cc99922 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:56.912706 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08169f1-e1ee-4c22-adba-102a0ffa7218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:56.913671 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08169f1-e1ee-4c22-adba-102a0ffa7218 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:56.913951 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08169f1-e1ee-4c22-adba-102a0ffa7218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:56.914223 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08169f1-e1ee-4c22-adba-102a0ffa7218 admin admin] PATH_INFO: `/v3/projects/ca51280a5d7547b8a5b3016461c1d432/users/ed7321b77d954ceaaf14ca97c82be6a0/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:56.931338 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08169f1-e1ee-4c22-adba-102a0ffa7218 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca51280a5d7547b8a5b3016461c1d432, user_id=ed7321b77d954ceaaf14ca97c82be6a0, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:56.934307 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08169f1-e1ee-4c22-adba-102a0ffa7218 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:56.948713 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1111/2220] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:56 2022] PUT /identity/v3/projects/ca51280a5d7547b8a5b3016461c1d432/users/ed7321b77d954ceaaf14ca97c82be6a0/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:56.960870 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c9d3a-c115-4ec5-bac0-bc6f8b5f4db5 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:56.961254 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c9d3a-c115-4ec5-bac0-bc6f8b5f4db5 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:56.961611 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c9d3a-c115-4ec5-bac0-bc6f8b5f4db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:57.005811 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-832c9d3a-c115-4ec5-bac0-bc6f8b5f4db5 None None] MFA Rules not processed for user `ed7321b77d954ceaaf14ca97c82be6a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:57.007116 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-832c9d3a-c115-4ec5-bac0-bc6f8b5f4db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:57.080765 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1110/2221] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:57.092920 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08169f1-e1ee-4c22-adba-102a0ffa7218 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:57.094528 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a08169f1-e1ee-4c22-adba-102a0ffa7218 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:57.151842 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08169f1-e1ee-4c22-adba-102a0ffa7218 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:57.155836 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7193a73-f63d-4f4f-a6cc-57605aca78a0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f10214b2bb4eafb3dff7adf71b9bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:57.156488 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7193a73-f63d-4f4f-a6cc-57605aca78a0 service glance] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:57.156770 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7193a73-f63d-4f4f-a6cc-57605aca78a0 service glance] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:57.157046 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7193a73-f63d-4f4f-a6cc-57605aca78a0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:57.167114 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7193a73-f63d-4f4f-a6cc-57605aca78a0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:57.168592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7193a73-f63d-4f4f-a6cc-57605aca78a0 service glance] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:57.178393 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1112/2222] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:58.065290 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c9d3a-c115-4ec5-bac0-bc6f8b5f4db5 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:58.067472 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-832c9d3a-c115-4ec5-bac0-bc6f8b5f4db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:58.157777 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c9d3a-c115-4ec5-bac0-bc6f8b5f4db5 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:58.163375 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca7403e-6b15-4204-b2f3-6be78d1fbac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:58.165095 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7403e-6b15-4204-b2f3-6be78d1fbac4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:58.165095 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7403e-6b15-4204-b2f3-6be78d1fbac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:58.165095 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7403e-6b15-4204-b2f3-6be78d1fbac4 admin admin] PATH_INFO: `/v3/users/96273a3f5e974eacae5b0e42c9ab0df5` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:58.167032 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca7403e-6b15-4204-b2f3-6be78d1fbac4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96273a3f5e974eacae5b0e42c9ab0df5)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:58.169219 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca7403e-6b15-4204-b2f3-6be78d1fbac4 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:58.239027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-4ca7403e-6b15-4204-b2f3-6be78d1fbac4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96273a3f5e974eacae5b0e42c9ab0df5'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:58.240305 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-4ca7403e-6b15-4204-b2f3-6be78d1fbac4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96273a3f5e974eacae5b0e42c9ab0df5'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:58.254970 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-4ca7403e-6b15-4204-b2f3-6be78d1fbac4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96273a3f5e974eacae5b0e42c9ab0df5'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:58.264481 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-4ca7403e-6b15-4204-b2f3-6be78d1fbac4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96273a3f5e974eacae5b0e42c9ab0df5'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:58.276373 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:58.281271 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-4ca7403e-6b15-4204-b2f3-6be78d1fbac4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96273a3f5e974eacae5b0e42c9ab0df5'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:58.288247 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1111/2223] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:58 2022] DELETE /identity/v3/users/96273a3f5e974eacae5b0e42c9ab0df5 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:58.303668 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313c01f-2513-4df5-8c0b-cc3920e79052 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:58.304139 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313c01f-2513-4df5-8c0b-cc3920e79052 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:58.304812 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313c01f-2513-4df5-8c0b-cc3920e79052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:58.331021 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-9313c01f-2513-4df5-8c0b-cc3920e79052 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:58.332336 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-9313c01f-2513-4df5-8c0b-cc3920e79052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:58.370706 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:58.404282 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1113/2224] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:44:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:58.414289 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca7403e-6b15-4204-b2f3-6be78d1fbac4 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:58.415749 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-4ca7403e-6b15-4204-b2f3-6be78d1fbac4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:58.481400 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca7403e-6b15-4204-b2f3-6be78d1fbac4 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:58.485131 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867726e-4305-4041-93cd-8db5f53e6018 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:58.485818 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867726e-4305-4041-93cd-8db5f53e6018 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:58.486099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867726e-4305-4041-93cd-8db5f53e6018 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:58.486372 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867726e-4305-4041-93cd-8db5f53e6018 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:58.496590 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5867726e-4305-4041-93cd-8db5f53e6018 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:58.498008 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5867726e-4305-4041-93cd-8db5f53e6018 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:58.508957 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1112/2225] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:58.811723 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9313c01f-2513-4df5-8c0b-cc3920e79052 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:58.813305 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-9313c01f-2513-4df5-8c0b-cc3920e79052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:58.839180 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9313c01f-2513-4df5-8c0b-cc3920e79052 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:58.844352 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ff44f-be82-440b-8528-3ac1f21365e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:58.845271 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ff44f-be82-440b-8528-3ac1f21365e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:58.845596 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ff44f-be82-440b-8528-3ac1f21365e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:58.845898 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ff44f-be82-440b-8528-3ac1f21365e5 admin admin] PATH_INFO: `/v3/projects/ef8cf056c69f4f6da196fd17428490a6` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:58.847178 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7ff44f-be82-440b-8528-3ac1f21365e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef8cf056c69f4f6da196fd17428490a6)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:58.851968 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7ff44f-be82-440b-8528-3ac1f21365e5 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:58.914561 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3a7ff44f-be82-440b-8528-3ac1f21365e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef8cf056c69f4f6da196fd17428490a6', 'request_id': 'req-3a7ff44f-be82-440b-8528-3ac1f21365e5'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:58.923234 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3a7ff44f-be82-440b-8528-3ac1f21365e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef8cf056c69f4f6da196fd17428490a6', 'request_id': 'req-3a7ff44f-be82-440b-8528-3ac1f21365e5'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:58.930368 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3a7ff44f-be82-440b-8528-3ac1f21365e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef8cf056c69f4f6da196fd17428490a6', 'request_id': 'req-3a7ff44f-be82-440b-8528-3ac1f21365e5'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:58.943999 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3a7ff44f-be82-440b-8528-3ac1f21365e5 admin admin] The token cache is being invalidate because project ef8cf056c69f4f6da196fd17428490a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:58.944367 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3a7ff44f-be82-440b-8528-3ac1f21365e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:58.945131 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3a7ff44f-be82-440b-8528-3ac1f21365e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:58.947648 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1114/2226] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:58 2022] DELETE /identity/v3/projects/ef8cf056c69f4f6da196fd17428490a6 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:05.676829 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cdaa5a-5f44-417d-ac63-5ab8e81ed99b None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:05.677349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cdaa5a-5f44-417d-ac63-5ab8e81ed99b None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:05.677442 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cdaa5a-5f44-417d-ac63-5ab8e81ed99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:05.707676 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-40cdaa5a-5f44-417d-ac63-5ab8e81ed99b None None] MFA Rules not processed for user `bf393488726e49338e70017f5cb7c4be`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:05.709612 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-40cdaa5a-5f44-417d-ac63-5ab8e81ed99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:05.782345 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1113/2227] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Jun 15 14:45:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:05.827679 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ff44f-be82-440b-8528-3ac1f21365e5 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:05.829183 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-3a7ff44f-be82-440b-8528-3ac1f21365e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:05.922811 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ff44f-be82-440b-8528-3ac1f21365e5 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:05.929266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be214c-89c9-4a61-a666-37bddca4e14f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '641a81b3140f47119b5c2323a97a6681', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:05.930143 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be214c-89c9-4a61-a666-37bddca4e14f service swift] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:05.930430 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be214c-89c9-4a61-a666-37bddca4e14f service swift] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:05.930703 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be214c-89c9-4a61-a666-37bddca4e14f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:05.940598 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87be214c-89c9-4a61-a666-37bddca4e14f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:05.942350 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87be214c-89c9-4a61-a666-37bddca4e14f service swift] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:05.952316 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1115/2228] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Jun 15 14:45:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:45:06.639854 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d566f5-58ff-4fd1-a9a0-103c109fac99 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:06.640147 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d566f5-58ff-4fd1-a9a0-103c109fac99 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:06.640428 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d566f5-58ff-4fd1-a9a0-103c109fac99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:06.658805 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-75d566f5-58ff-4fd1-a9a0-103c109fac99 None None] MFA Rules not processed for user `bf393488726e49338e70017f5cb7c4be`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:06.660408 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-75d566f5-58ff-4fd1-a9a0-103c109fac99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:06.669142 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1114/2229] 10.222.0.45 () {58 vars in 967 bytes} [Wed Jun 15 14:45:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:06.681991 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be214c-89c9-4a61-a666-37bddca4e14f service swift] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:06.693308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be214c-89c9-4a61-a666-37bddca4e14f service swift] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:06.699728 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1fdebe-d13e-4509-80a8-41227aabce46 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '641a81b3140f47119b5c2323a97a6681', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:06.700782 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1fdebe-d13e-4509-80a8-41227aabce46 service swift] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:06.701231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1fdebe-d13e-4509-80a8-41227aabce46 service swift] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:06.701711 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1fdebe-d13e-4509-80a8-41227aabce46 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:06.713433 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1fdebe-d13e-4509-80a8-41227aabce46 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:06.715083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1fdebe-d13e-4509-80a8-41227aabce46 service swift] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:06.724230 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1116/2230] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Jun 15 14:45:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:45:06.952350 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e301bc-e2d9-4189-b46f-69deef049e74 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:06.952788 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e301bc-e2d9-4189-b46f-69deef049e74 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:06.952908 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e301bc-e2d9-4189-b46f-69deef049e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:06.972415 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-b1e301bc-e2d9-4189-b46f-69deef049e74 None None] MFA Rules not processed for user `bf393488726e49338e70017f5cb7c4be`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:06.973614 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b1e301bc-e2d9-4189-b46f-69deef049e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:06.981820 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1115/2231] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Jun 15 14:45:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:06.996281 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1fdebe-d13e-4509-80a8-41227aabce46 service swift] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:07.007374 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1fdebe-d13e-4509-80a8-41227aabce46 service swift] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:07.012606 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4caa533-ce50-4d4e-9fdf-1b9643946d5d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '641a81b3140f47119b5c2323a97a6681', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:07.013382 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4caa533-ce50-4d4e-9fdf-1b9643946d5d service swift] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:07.013651 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4caa533-ce50-4d4e-9fdf-1b9643946d5d service swift] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:07.013935 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4caa533-ce50-4d4e-9fdf-1b9643946d5d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:07.022576 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4caa533-ce50-4d4e-9fdf-1b9643946d5d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:07.024423 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4caa533-ce50-4d4e-9fdf-1b9643946d5d service swift] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:07.033632 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1117/2232] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Jun 15 14:45:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:45:07.265945 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e301bc-e2d9-4189-b46f-69deef049e74 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:07.267791 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b1e301bc-e2d9-4189-b46f-69deef049e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:07.342393 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e301bc-e2d9-4189-b46f-69deef049e74 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:07.347936 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95107f36-5b9b-45c0-87bc-b8cfc7f305d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:07.349501 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95107f36-5b9b-45c0-87bc-b8cfc7f305d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:07.349783 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95107f36-5b9b-45c0-87bc-b8cfc7f305d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:07.350054 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95107f36-5b9b-45c0-87bc-b8cfc7f305d2 admin admin] PATH_INFO: `/v3/users/6ff94fb079d24ddc882d8f8be276dbb4` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:07.351970 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95107f36-5b9b-45c0-87bc-b8cfc7f305d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ff94fb079d24ddc882d8f8be276dbb4)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:07.354046 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95107f36-5b9b-45c0-87bc-b8cfc7f305d2 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:07.371030 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:07.410862 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-95107f36-5b9b-45c0-87bc-b8cfc7f305d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ff94fb079d24ddc882d8f8be276dbb4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:07.411672 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-95107f36-5b9b-45c0-87bc-b8cfc7f305d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ff94fb079d24ddc882d8f8be276dbb4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:07.423885 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-95107f36-5b9b-45c0-87bc-b8cfc7f305d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ff94fb079d24ddc882d8f8be276dbb4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:07.434644 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-95107f36-5b9b-45c0-87bc-b8cfc7f305d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ff94fb079d24ddc882d8f8be276dbb4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:07.437068 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e4919-a6a4-473f-8633-bcbdca96d0f0 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:07.437435 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e4919-a6a4-473f-8633-bcbdca96d0f0 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:07.437764 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e4919-a6a4-473f-8633-bcbdca96d0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:07.452056 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-95107f36-5b9b-45c0-87bc-b8cfc7f305d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ff94fb079d24ddc882d8f8be276dbb4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:07.462764 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1116/2233] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:45:07 2022] DELETE /identity/v3/users/6ff94fb079d24ddc882d8f8be276dbb4 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:07.464959 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a0667-5497-4817-a8b7-41acea7fe943 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:07.465336 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a0667-5497-4817-a8b7-41acea7fe943 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:07.465691 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a0667-5497-4817-a8b7-41acea7fe943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:07.471751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-924e4919-a6a4-473f-8633-bcbdca96d0f0 None None] MFA Rules not processed for user `bf393488726e49338e70017f5cb7c4be`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:07.474592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-924e4919-a6a4-473f-8633-bcbdca96d0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:07.485924 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-d87a0667-5497-4817-a8b7-41acea7fe943 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:07.487171 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d87a0667-5497-4817-a8b7-41acea7fe943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:07.499719 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:07.545887 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1118/2234] 10.222.0.45 () {58 vars in 967 bytes} [Wed Jun 15 14:45:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:07.559120 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e4919-a6a4-473f-8633-bcbdca96d0f0 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:07.560474 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-924e4919-a6a4-473f-8633-bcbdca96d0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:07.564530 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1117/2235] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:45:07.575522 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a0667-5497-4817-a8b7-41acea7fe943 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:07.576904 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d87a0667-5497-4817-a8b7-41acea7fe943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:07.640144 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e4919-a6a4-473f-8633-bcbdca96d0f0 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:07.643556 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a0667-5497-4817-a8b7-41acea7fe943 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:07.645383 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526cd5b2-2850-4c21-ab4d-89e6458c2637 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '641a81b3140f47119b5c2323a97a6681', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:07.646177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cd5b2-2850-4c21-ab4d-89e6458c2637 service swift] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:07.646591 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cd5b2-2850-4c21-ab4d-89e6458c2637 service swift] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:07.646875 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cd5b2-2850-4c21-ab4d-89e6458c2637 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:07.647337 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c12164-fe6e-4c82-beae-1c5af5a6df02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:07.647983 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c12164-fe6e-4c82-beae-1c5af5a6df02 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:07.648271 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c12164-fe6e-4c82-beae-1c5af5a6df02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:07.648552 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c12164-fe6e-4c82-beae-1c5af5a6df02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:07.655973 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526cd5b2-2850-4c21-ab4d-89e6458c2637 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:07.657727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526cd5b2-2850-4c21-ab4d-89e6458c2637 service swift] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:07.659212 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c12164-fe6e-4c82-beae-1c5af5a6df02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:07.660714 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c12164-fe6e-4c82-beae-1c5af5a6df02 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:07.667305 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1119/2236] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Jun 15 14:45:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 15 14:45:07.671221 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1118/2237] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:45:08.201742 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526cd5b2-2850-4c21-ab4d-89e6458c2637 service swift] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:08.203234 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-526cd5b2-2850-4c21-ab4d-89e6458c2637 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:08.218490 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526cd5b2-2850-4c21-ab4d-89e6458c2637 service swift] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:08.224766 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eba797-603d-405d-ac60-a68883570b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:08.226291 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eba797-603d-405d-ac60-a68883570b68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:08.226955 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eba797-603d-405d-ac60-a68883570b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:08.227525 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eba797-603d-405d-ac60-a68883570b68 admin admin] PATH_INFO: `/v3/projects/4b8b91c41d98497a849e69acf55912a8` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:08.234397 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eba797-603d-405d-ac60-a68883570b68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b8b91c41d98497a849e69acf55912a8)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:08.236668 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eba797-603d-405d-ac60-a68883570b68 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:08.291317 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-37eba797-603d-405d-ac60-a68883570b68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b8b91c41d98497a849e69acf55912a8', 'request_id': 'req-37eba797-603d-405d-ac60-a68883570b68'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:08.299842 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-37eba797-603d-405d-ac60-a68883570b68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b8b91c41d98497a849e69acf55912a8', 'request_id': 'req-37eba797-603d-405d-ac60-a68883570b68'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:08.315047 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-37eba797-603d-405d-ac60-a68883570b68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b8b91c41d98497a849e69acf55912a8', 'request_id': 'req-37eba797-603d-405d-ac60-a68883570b68'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:08.326510 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-37eba797-603d-405d-ac60-a68883570b68 admin admin] The token cache is being invalidate because project 4b8b91c41d98497a849e69acf55912a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:45:08.326799 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-37eba797-603d-405d-ac60-a68883570b68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:08.327539 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-37eba797-603d-405d-ac60-a68883570b68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:08.329094 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1120/2238] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:45:08 2022] DELETE /identity/v3/projects/4b8b91c41d98497a849e69acf55912a8 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:08.332948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c12164-fe6e-4c82-beae-1c5af5a6df02 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:08.345065 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c12164-fe6e-4c82-beae-1c5af5a6df02 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:08.378717 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:08.410620 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e6114-8a1e-4d91-bab3-5e60a086c0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:08.412182 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e6114-8a1e-4d91-bab3-5e60a086c0dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:08.412551 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e6114-8a1e-4d91-bab3-5e60a086c0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:08.412888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e6114-8a1e-4d91-bab3-5e60a086c0dd admin admin] PATH_INFO: `/v3/users/cbe5f30386f24344822043be00429b27` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:08.414200 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e6114-8a1e-4d91-bab3-5e60a086c0dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbe5f30386f24344822043be00429b27)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:08.416438 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e6114-8a1e-4d91-bab3-5e60a086c0dd admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:08.473311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-be3e6114-8a1e-4d91-bab3-5e60a086c0dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbe5f30386f24344822043be00429b27'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:08.474568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-be3e6114-8a1e-4d91-bab3-5e60a086c0dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbe5f30386f24344822043be00429b27'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:08.487494 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-be3e6114-8a1e-4d91-bab3-5e60a086c0dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbe5f30386f24344822043be00429b27'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:08.496440 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-be3e6114-8a1e-4d91-bab3-5e60a086c0dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbe5f30386f24344822043be00429b27'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:08.503664 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-be3e6114-8a1e-4d91-bab3-5e60a086c0dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbe5f30386f24344822043be00429b27'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:08.514858 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1119/2239] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:45:08 2022] DELETE /identity/v3/users/cbe5f30386f24344822043be00429b27 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:08.895425 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eba797-603d-405d-ac60-a68883570b68 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:08.897401 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-37eba797-603d-405d-ac60-a68883570b68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:08.922977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eba797-603d-405d-ac60-a68883570b68 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:08.963343 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:08.982570 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76410db2-88df-499d-9252-97e82724ded5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:08.983477 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76410db2-88df-499d-9252-97e82724ded5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:08.983766 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76410db2-88df-499d-9252-97e82724ded5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:08.984047 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76410db2-88df-499d-9252-97e82724ded5 admin admin] PATH_INFO: `/v3/projects/0b55d6495ef849ffb8e41a202537feb1` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:08.985236 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76410db2-88df-499d-9252-97e82724ded5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b55d6495ef849ffb8e41a202537feb1)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:08.987193 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76410db2-88df-499d-9252-97e82724ded5 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:09.033729 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-76410db2-88df-499d-9252-97e82724ded5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b55d6495ef849ffb8e41a202537feb1', 'request_id': 'req-76410db2-88df-499d-9252-97e82724ded5'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:09.042003 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-76410db2-88df-499d-9252-97e82724ded5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b55d6495ef849ffb8e41a202537feb1', 'request_id': 'req-76410db2-88df-499d-9252-97e82724ded5'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:09.048774 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-76410db2-88df-499d-9252-97e82724ded5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b55d6495ef849ffb8e41a202537feb1', 'request_id': 'req-76410db2-88df-499d-9252-97e82724ded5'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:09.059587 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-76410db2-88df-499d-9252-97e82724ded5 admin admin] The token cache is being invalidate because project 0b55d6495ef849ffb8e41a202537feb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:45:09.059876 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-76410db2-88df-499d-9252-97e82724ded5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:09.060633 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-76410db2-88df-499d-9252-97e82724ded5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:09.062444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1121/2240] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:45:08 2022] DELETE /identity/v3/projects/0b55d6495ef849ffb8e41a202537feb1 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:09.067847 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997af7a1-78e7-4fa1-8d91-a067272bc259 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:09.068180 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997af7a1-78e7-4fa1-8d91-a067272bc259 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:09.068516 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997af7a1-78e7-4fa1-8d91-a067272bc259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:09.086836 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-997af7a1-78e7-4fa1-8d91-a067272bc259 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:09.088082 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-997af7a1-78e7-4fa1-8d91-a067272bc259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:09.152816 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1120/2241] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:45:09.161033 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ee550-4591-476d-9b33-5a285ae2e05a None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:09.161819 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ee550-4591-476d-9b33-5a285ae2e05a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:09.162098 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ee550-4591-476d-9b33-5a285ae2e05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:09.182780 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-e96ee550-4591-476d-9b33-5a285ae2e05a None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:09.184087 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e96ee550-4591-476d-9b33-5a285ae2e05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:09.192169 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1122/2242] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:09.196531 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997af7a1-78e7-4fa1-8d91-a067272bc259 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:09.206908 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997af7a1-78e7-4fa1-8d91-a067272bc259 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:09.211913 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f61ae-4a3d-4cc1-bd39-4008e2f5c5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:09.212500 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f61ae-4a3d-4cc1-bd39-4008e2f5c5f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:09.212786 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f61ae-4a3d-4cc1-bd39-4008e2f5c5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:09.213066 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f61ae-4a3d-4cc1-bd39-4008e2f5c5f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:09.213828 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f61ae-4a3d-4cc1-bd39-4008e2f5c5f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:45:09.214139 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f61ae-4a3d-4cc1-bd39-4008e2f5c5f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:09.215794 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f61ae-4a3d-4cc1-bd39-4008e2f5c5f5 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:09.228334 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1121/2243] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:45:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:45:09.233182 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96ee550-4591-476d-9b33-5a285ae2e05a None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:09.244915 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96ee550-4591-476d-9b33-5a285ae2e05a None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:09.249629 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fba5eb-0aa9-4945-a404-1b060593fbed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:09.250295 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fba5eb-0aa9-4945-a404-1b060593fbed admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:09.250583 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fba5eb-0aa9-4945-a404-1b060593fbed admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:09.250864 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fba5eb-0aa9-4945-a404-1b060593fbed admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:09.251598 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fba5eb-0aa9-4945-a404-1b060593fbed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:09.253619 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fba5eb-0aa9-4945-a404-1b060593fbed admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:09.285058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1123/2244] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:45:09 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:45:09.287197 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f61ae-4a3d-4cc1-bd39-4008e2f5c5f5 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:09.300304 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f61ae-4a3d-4cc1-bd39-4008e2f5c5f5 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:09.339661 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:09.361811 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e4167c-8b56-4bac-ac92-f9a10fad38e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:09.362495 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e4167c-8b56-4bac-ac92-f9a10fad38e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:09.362789 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e4167c-8b56-4bac-ac92-f9a10fad38e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:09.363111 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e4167c-8b56-4bac-ac92-f9a10fad38e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:09.363845 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e4167c-8b56-4bac-ac92-f9a10fad38e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:09.366454 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e4167c-8b56-4bac-ac92-f9a10fad38e7 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:09.386052 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1122/2245] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:45:09 2022] POST /identity/v3/users => generated 284 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:45:09.390202 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fba5eb-0aa9-4945-a404-1b060593fbed admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:09.400906 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fba5eb-0aa9-4945-a404-1b060593fbed admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:09.406275 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b927f4e-f059-4e3e-84fd-958f49b5b134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:09.406962 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b927f4e-f059-4e3e-84fd-958f49b5b134 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:09.407272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b927f4e-f059-4e3e-84fd-958f49b5b134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:09.407548 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b927f4e-f059-4e3e-84fd-958f49b5b134 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:09.408297 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b927f4e-f059-4e3e-84fd-958f49b5b134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:09.410002 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b927f4e-f059-4e3e-84fd-958f49b5b134 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:09.421661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1124/2246] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:09.425819 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e4167c-8b56-4bac-ac92-f9a10fad38e7 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:09.436644 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e4167c-8b56-4bac-ac92-f9a10fad38e7 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:09.441263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1deeba-f00e-4445-aba8-d6a597877dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:09.441913 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1deeba-f00e-4445-aba8-d6a597877dc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:09.442200 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1deeba-f00e-4445-aba8-d6a597877dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:09.442498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1deeba-f00e-4445-aba8-d6a597877dc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:09.443317 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1deeba-f00e-4445-aba8-d6a597877dc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:09.444905 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1deeba-f00e-4445-aba8-d6a597877dc5 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:09.455279 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1123/2247] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:09.459557 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b927f4e-f059-4e3e-84fd-958f49b5b134 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:09.469769 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b927f4e-f059-4e3e-84fd-958f49b5b134 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:09.475745 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb6434-ebbd-46fd-b61c-d82cdb9eb018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:09.476685 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb6434-ebbd-46fd-b61c-d82cdb9eb018 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:09.476970 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb6434-ebbd-46fd-b61c-d82cdb9eb018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:09.477246 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb6434-ebbd-46fd-b61c-d82cdb9eb018 admin admin] PATH_INFO: `/v3/projects/d1f0624454af4e53821b60c4ffe41d2d/users/1d839135353f4856b54758b3ece46178/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:09.492632 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb6434-ebbd-46fd-b61c-d82cdb9eb018 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1f0624454af4e53821b60c4ffe41d2d, user_id=1d839135353f4856b54758b3ece46178, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:09.495308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb6434-ebbd-46fd-b61c-d82cdb9eb018 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:09.507996 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1125/2248] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:45:09 2022] PUT /identity/v3/projects/d1f0624454af4e53821b60c4ffe41d2d/users/1d839135353f4856b54758b3ece46178/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:09.521508 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ff30d-f047-4389-b3e7-cfb302f7cb60 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:09.521807 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ff30d-f047-4389-b3e7-cfb302f7cb60 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:09.522098 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ff30d-f047-4389-b3e7-cfb302f7cb60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:09.549718 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-ae6ff30d-f047-4389-b3e7-cfb302f7cb60 None None] MFA Rules not processed for user `1d839135353f4856b54758b3ece46178`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:09.550937 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ae6ff30d-f047-4389-b3e7-cfb302f7cb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:09.623012 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1124/2249] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:09 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:09.627746 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb6434-ebbd-46fd-b61c-d82cdb9eb018 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:09.690960 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb6434-ebbd-46fd-b61c-d82cdb9eb018 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:09.695536 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2fd0f-1e0b-42ab-a644-c68a943b39a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:09.696191 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2fd0f-1e0b-42ab-a644-c68a943b39a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:09.696471 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2fd0f-1e0b-42ab-a644-c68a943b39a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:09.696745 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2fd0f-1e0b-42ab-a644-c68a943b39a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:09.697430 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e2fd0f-1e0b-42ab-a644-c68a943b39a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:09.699387 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e2fd0f-1e0b-42ab-a644-c68a943b39a5 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:09.723103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:09.746283 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1126/2250] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:45:09 2022] POST /identity/v3/projects => generated 369 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:45:09.747719 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ff30d-f047-4389-b3e7-cfb302f7cb60 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:09.758413 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ff30d-f047-4389-b3e7-cfb302f7cb60 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:09.820519 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69805958-bd2f-49f6-8fa3-5ecaf2e79051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:09.821193 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69805958-bd2f-49f6-8fa3-5ecaf2e79051 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:09.821476 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69805958-bd2f-49f6-8fa3-5ecaf2e79051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:09.821751 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69805958-bd2f-49f6-8fa3-5ecaf2e79051 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:09.822472 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69805958-bd2f-49f6-8fa3-5ecaf2e79051 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:09.824454 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69805958-bd2f-49f6-8fa3-5ecaf2e79051 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:09.849499 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1125/2251] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:45:09 2022] POST /identity/v3/users => generated 291 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:45:09.853889 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2fd0f-1e0b-42ab-a644-c68a943b39a5 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:09.874585 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2fd0f-1e0b-42ab-a644-c68a943b39a5 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:09.880711 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69805958-bd2f-49f6-8fa3-5ecaf2e79051 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:09.882442 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-69805958-bd2f-49f6-8fa3-5ecaf2e79051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:09.883621 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a4093-c2d2-43df-bf33-be3493fb1f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:09.884407 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a4093-c2d2-43df-bf33-be3493fb1f1b admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:09.884693 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a4093-c2d2-43df-bf33-be3493fb1f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:09.884974 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a4093-c2d2-43df-bf33-be3493fb1f1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:09.886388 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229a4093-c2d2-43df-bf33-be3493fb1f1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:09.888107 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229a4093-c2d2-43df-bf33-be3493fb1f1b admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:09.897309 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69805958-bd2f-49f6-8fa3-5ecaf2e79051 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:09.909200 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1127/2252] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:09 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:09.910185 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e089ae-3716-416d-8ede-f25b1dc5910e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:09.910185 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e089ae-3716-416d-8ede-f25b1dc5910e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:09.910185 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e089ae-3716-416d-8ede-f25b1dc5910e admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:09.910185 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e089ae-3716-416d-8ede-f25b1dc5910e admin admin] PATH_INFO: `/v3/users/453c056ef8674822a0fc93c1c6675664` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:09.910932 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a4093-c2d2-43df-bf33-be3493fb1f1b admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:09.912544 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e089ae-3716-416d-8ede-f25b1dc5910e admin admin] RBAC: Authorizing `identity:delete_user(user_id=453c056ef8674822a0fc93c1c6675664)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:09.914512 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e089ae-3716-416d-8ede-f25b1dc5910e admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:09.929261 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a4093-c2d2-43df-bf33-be3493fb1f1b admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:09.934131 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455e360-01f4-4d80-84e8-736ebb4ffe5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:09.935121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455e360-01f4-4d80-84e8-736ebb4ffe5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:09.935566 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455e360-01f4-4d80-84e8-736ebb4ffe5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:09.935987 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455e360-01f4-4d80-84e8-736ebb4ffe5b admin admin] PATH_INFO: `/v3/projects/2d19e07ec6d541b3a55064f898424d71/users/adcb5bfc291a4eefb9b4fff6f140c782/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:09.956437 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b455e360-01f4-4d80-84e8-736ebb4ffe5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d19e07ec6d541b3a55064f898424d71, user_id=adcb5bfc291a4eefb9b4fff6f140c782, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:09.962957 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b455e360-01f4-4d80-84e8-736ebb4ffe5b admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:09.964361 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:INSERT=6,keystone:DELETE=5 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:09.992064 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1128/2253] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:45:09 2022] PUT /identity/v3/projects/2d19e07ec6d541b3a55064f898424d71/users/adcb5bfc291a4eefb9b4fff6f140c782/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:09.993457 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455e360-01f4-4d80-84e8-736ebb4ffe5b admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:09.994394 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c7e089ae-3716-416d-8ede-f25b1dc5910e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '453c056ef8674822a0fc93c1c6675664'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:09.996868 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c7e089ae-3716-416d-8ede-f25b1dc5910e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '453c056ef8674822a0fc93c1c6675664'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:10.014074 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c7e089ae-3716-416d-8ede-f25b1dc5910e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '453c056ef8674822a0fc93c1c6675664'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:10.022727 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c7e089ae-3716-416d-8ede-f25b1dc5910e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '453c056ef8674822a0fc93c1c6675664'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:10.030158 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c7e089ae-3716-416d-8ede-f25b1dc5910e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '453c056ef8674822a0fc93c1c6675664'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:10.036700 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1126/2254] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:45:09 2022] DELETE /identity/v3/users/453c056ef8674822a0fc93c1c6675664 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:10.040905 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039d7c9-8660-4b0d-9473-d06805494e8c None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:10.041223 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039d7c9-8660-4b0d-9473-d06805494e8c None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:10.041503 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039d7c9-8660-4b0d-9473-d06805494e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:10.059765 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-b039d7c9-8660-4b0d-9473-d06805494e8c None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:10.061165 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b039d7c9-8660-4b0d-9473-d06805494e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:10.070427 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455e360-01f4-4d80-84e8-736ebb4ffe5b admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:10.124094 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1127/2255] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:45:10.129735 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b921a-b65c-420f-8259-c178a0b5e03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:10.131044 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b921a-b65c-420f-8259-c178a0b5e03c admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:10.131336 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b921a-b65c-420f-8259-c178a0b5e03c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:10.131618 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b921a-b65c-420f-8259-c178a0b5e03c admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:10.132376 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111b921a-b65c-420f-8259-c178a0b5e03c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:10.134261 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111b921a-b65c-420f-8259-c178a0b5e03c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:10.134592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039d7c9-8660-4b0d-9473-d06805494e8c None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:10.136004 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b039d7c9-8660-4b0d-9473-d06805494e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:10.149384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1129/2256] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:09 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:10.153930 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b921a-b65c-420f-8259-c178a0b5e03c admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:10.155656 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-111b921a-b65c-420f-8259-c178a0b5e03c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:10.167916 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b921a-b65c-420f-8259-c178a0b5e03c admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:10.172585 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d4aa0c-0301-45e6-acf7-44517e58d663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:10.173458 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4aa0c-0301-45e6-acf7-44517e58d663 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:10.173737 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4aa0c-0301-45e6-acf7-44517e58d663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:10.174011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4aa0c-0301-45e6-acf7-44517e58d663 admin admin] PATH_INFO: `/v3/projects/2d19e07ec6d541b3a55064f898424d71/users/adcb5bfc291a4eefb9b4fff6f140c782/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:10.176034 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d4aa0c-0301-45e6-acf7-44517e58d663 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d19e07ec6d541b3a55064f898424d71, user_id=adcb5bfc291a4eefb9b4fff6f140c782, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:10.178724 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d4aa0c-0301-45e6-acf7-44517e58d663 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:10.191374 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1130/2257] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:45:10 2022] PUT /identity/v3/projects/2d19e07ec6d541b3a55064f898424d71/users/adcb5bfc291a4eefb9b4fff6f140c782/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:45:10.198914 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039d7c9-8660-4b0d-9473-d06805494e8c None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:10.202451 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9de74-ea53-4bfb-9eb5-64f1852e7538 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:10.202740 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9de74-ea53-4bfb-9eb5-64f1852e7538 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:10.203056 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9de74-ea53-4bfb-9eb5-64f1852e7538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:10.230576 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-76d9de74-ea53-4bfb-9eb5-64f1852e7538 None None] MFA Rules not processed for user `adcb5bfc291a4eefb9b4fff6f140c782`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:10.231740 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-76d9de74-ea53-4bfb-9eb5-64f1852e7538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:10.257649 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f19caf-48ec-4bf4-8aca-ecce7cc9bc1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:10.258306 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f19caf-48ec-4bf4-8aca-ecce7cc9bc1e service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:10.258595 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f19caf-48ec-4bf4-8aca-ecce7cc9bc1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:10.259235 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f19caf-48ec-4bf4-8aca-ecce7cc9bc1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:10.281553 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:10.282545 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:10.317190 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1131/2258] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:10 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:45:10.331274 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9de74-ea53-4bfb-9eb5-64f1852e7538 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:10.332905 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-76d9de74-ea53-4bfb-9eb5-64f1852e7538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:10.345171 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f19caf-48ec-4bf4-8aca-ecce7cc9bc1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:10.346621 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f19caf-48ec-4bf4-8aca-ecce7cc9bc1e service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:10.358579 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1128/2259] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:45:10.397949 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9de74-ea53-4bfb-9eb5-64f1852e7538 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:10.401851 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9957842-32e0-4042-9fc5-90271c3433e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:10.402507 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9957842-32e0-4042-9fc5-90271c3433e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:10.402791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9957842-32e0-4042-9fc5-90271c3433e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:10.403580 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9957842-32e0-4042-9fc5-90271c3433e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:10.413614 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9957842-32e0-4042-9fc5-90271c3433e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:10.415186 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9957842-32e0-4042-9fc5-90271c3433e4 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:10.427668 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1132/2260] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:10 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:45:10.550464 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f19caf-48ec-4bf4-8aca-ecce7cc9bc1e service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:10.552462 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-99f19caf-48ec-4bf4-8aca-ecce7cc9bc1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:10.569458 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f19caf-48ec-4bf4-8aca-ecce7cc9bc1e service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:10.588023 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d3fe3-370b-4488-8d68-9b61402c32ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:10.588885 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d3fe3-370b-4488-8d68-9b61402c32ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:10.589168 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d3fe3-370b-4488-8d68-9b61402c32ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:10.589442 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d3fe3-370b-4488-8d68-9b61402c32ca admin admin] PATH_INFO: `/v3/projects/fae043d79ce040e0affb39e23162523a` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:10.591078 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2d3fe3-370b-4488-8d68-9b61402c32ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=fae043d79ce040e0affb39e23162523a)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:10.594192 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2d3fe3-370b-4488-8d68-9b61402c32ca admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:10.653947 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-5b2d3fe3-370b-4488-8d68-9b61402c32ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fae043d79ce040e0affb39e23162523a', 'request_id': 'req-5b2d3fe3-370b-4488-8d68-9b61402c32ca'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:10.663496 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-5b2d3fe3-370b-4488-8d68-9b61402c32ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fae043d79ce040e0affb39e23162523a', 'request_id': 'req-5b2d3fe3-370b-4488-8d68-9b61402c32ca'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:10.670455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-5b2d3fe3-370b-4488-8d68-9b61402c32ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fae043d79ce040e0affb39e23162523a', 'request_id': 'req-5b2d3fe3-370b-4488-8d68-9b61402c32ca'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:10.682140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-5b2d3fe3-370b-4488-8d68-9b61402c32ca admin admin] The token cache is being invalidate because project fae043d79ce040e0affb39e23162523a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:45:10.682483 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-5b2d3fe3-370b-4488-8d68-9b61402c32ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:10.682942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-5b2d3fe3-370b-4488-8d68-9b61402c32ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:10.684662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1129/2261] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:45:10 2022] DELETE /identity/v3/projects/fae043d79ce040e0affb39e23162523a => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:45:10.690011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aedb5a-67d1-4224-adaa-57da3cdcc4d7 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:10.690375 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aedb5a-67d1-4224-adaa-57da3cdcc4d7 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:10.690772 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aedb5a-67d1-4224-adaa-57da3cdcc4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:10.716078 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-10aedb5a-67d1-4224-adaa-57da3cdcc4d7 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:10.717789 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-10aedb5a-67d1-4224-adaa-57da3cdcc4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:10.789271 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1133/2262] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:10.797171 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ecc401-d405-4c0c-9de7-d61873023be5 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:10.797397 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ecc401-d405-4c0c-9de7-d61873023be5 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:10.797677 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ecc401-d405-4c0c-9de7-d61873023be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:10.821987 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-d2ecc401-d405-4c0c-9de7-d61873023be5 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:10.823350 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d2ecc401-d405-4c0c-9de7-d61873023be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:10.837807 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1130/2263] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:10.842000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aedb5a-67d1-4224-adaa-57da3cdcc4d7 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:10.852226 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aedb5a-67d1-4224-adaa-57da3cdcc4d7 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:10.856869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0527b177-9041-4020-9c77-dc738d0c076d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:10.857529 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527b177-9041-4020-9c77-dc738d0c076d admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:10.857816 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527b177-9041-4020-9c77-dc738d0c076d admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:10.858092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527b177-9041-4020-9c77-dc738d0c076d admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:10.858820 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0527b177-9041-4020-9c77-dc738d0c076d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:45:10.859169 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0527b177-9041-4020-9c77-dc738d0c076d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:10.860764 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0527b177-9041-4020-9c77-dc738d0c076d admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:10.873964 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1134/2264] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:45:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:45:10.881248 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ecc401-d405-4c0c-9de7-d61873023be5 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:10.892263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ecc401-d405-4c0c-9de7-d61873023be5 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:10.900520 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8724f-1c00-4f39-b5c5-2d53890af5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:10.901253 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8724f-1c00-4f39-b5c5-2d53890af5d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:10.901648 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8724f-1c00-4f39-b5c5-2d53890af5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:10.901991 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8724f-1c00-4f39-b5c5-2d53890af5d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:10.902744 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe8724f-1c00-4f39-b5c5-2d53890af5d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:10.905035 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe8724f-1c00-4f39-b5c5-2d53890af5d7 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:10.939659 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1131/2265] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:45:10 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:45:10.940886 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0527b177-9041-4020-9c77-dc738d0c076d admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:10.951499 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0527b177-9041-4020-9c77-dc738d0c076d admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:11.019760 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613988b0-241e-4fd6-9f5b-a314ae6d2f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:11.020357 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613988b0-241e-4fd6-9f5b-a314ae6d2f1f admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:11.021384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613988b0-241e-4fd6-9f5b-a314ae6d2f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:11.021625 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613988b0-241e-4fd6-9f5b-a314ae6d2f1f admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:11.022308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613988b0-241e-4fd6-9f5b-a314ae6d2f1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:11.024361 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613988b0-241e-4fd6-9f5b-a314ae6d2f1f admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:11.035210 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:11.048357 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1135/2266] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:45:10 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:45:11.053250 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8724f-1c00-4f39-b5c5-2d53890af5d7 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:11.067344 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8724f-1c00-4f39-b5c5-2d53890af5d7 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:11.073791 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca09ae7-2b26-4e21-9206-a58b4fe62026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:11.074469 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca09ae7-2b26-4e21-9206-a58b4fe62026 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:11.074755 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca09ae7-2b26-4e21-9206-a58b4fe62026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:11.075074 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca09ae7-2b26-4e21-9206-a58b4fe62026 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:11.075822 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca09ae7-2b26-4e21-9206-a58b4fe62026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:11.077417 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca09ae7-2b26-4e21-9206-a58b4fe62026 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:11.089315 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1132/2267] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:11.093064 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613988b0-241e-4fd6-9f5b-a314ae6d2f1f admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:11.103623 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613988b0-241e-4fd6-9f5b-a314ae6d2f1f admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:11.109928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e7384-87a0-422c-bfd5-3d889dbb5c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:11.110636 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e7384-87a0-422c-bfd5-3d889dbb5c8c admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:11.110919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e7384-87a0-422c-bfd5-3d889dbb5c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:11.111253 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e7384-87a0-422c-bfd5-3d889dbb5c8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:11.111986 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e7384-87a0-422c-bfd5-3d889dbb5c8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:11.113590 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e7384-87a0-422c-bfd5-3d889dbb5c8c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:11.126266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1136/2268] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:45:11.134128 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca09ae7-2b26-4e21-9206-a58b4fe62026 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:11.159463 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca09ae7-2b26-4e21-9206-a58b4fe62026 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:11.165098 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3fb84d-8dde-47cd-9db8-7419e0c95bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:11.165964 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3fb84d-8dde-47cd-9db8-7419e0c95bb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:11.166253 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3fb84d-8dde-47cd-9db8-7419e0c95bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:11.166532 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3fb84d-8dde-47cd-9db8-7419e0c95bb6 admin admin] PATH_INFO: `/v3/projects/3e38db34f53b4271ab5462a872e954ef/users/5d47f5d89bbb4d43844b1fb0794a3e69/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:11.184275 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3fb84d-8dde-47cd-9db8-7419e0c95bb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e38db34f53b4271ab5462a872e954ef, user_id=5d47f5d89bbb4d43844b1fb0794a3e69, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:11.187126 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3fb84d-8dde-47cd-9db8-7419e0c95bb6 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:11.204550 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1133/2269] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:45:11 2022] PUT /identity/v3/projects/3e38db34f53b4271ab5462a872e954ef/users/5d47f5d89bbb4d43844b1fb0794a3e69/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:45:11.214612 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3acbdd-71b7-431d-a4b4-50f1f3a37ced None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:11.214612 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3acbdd-71b7-431d-a4b4-50f1f3a37ced None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:11.215030 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3acbdd-71b7-431d-a4b4-50f1f3a37ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:11.237753 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-cd3acbdd-71b7-431d-a4b4-50f1f3a37ced None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:11.241305 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-cd3acbdd-71b7-431d-a4b4-50f1f3a37ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:11.308513 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1137/2270] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:11.321614 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3fb84d-8dde-47cd-9db8-7419e0c95bb6 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:11.322179 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-0f3fb84d-8dde-47cd-9db8-7419e0c95bb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:11.363878 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:11.386168 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3fb84d-8dde-47cd-9db8-7419e0c95bb6 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:11.390897 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d23199-dba5-4a04-ab64-fdc1720c3321 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:11.391766 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d23199-dba5-4a04-ab64-fdc1720c3321 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:11.392158 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d23199-dba5-4a04-ab64-fdc1720c3321 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:11.392498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d23199-dba5-4a04-ab64-fdc1720c3321 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:11.404179 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d23199-dba5-4a04-ab64-fdc1720c3321 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:11.405944 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d23199-dba5-4a04-ab64-fdc1720c3321 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:11.417231 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1134/2271] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:45:11.915319 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3acbdd-71b7-431d-a4b4-50f1f3a37ced None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:11.916841 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-cd3acbdd-71b7-431d-a4b4-50f1f3a37ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:11.929686 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3acbdd-71b7-431d-a4b4-50f1f3a37ced None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:11.934493 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2269c7d-dab1-41fa-9615-d5b92b9d3c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:11.935413 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2269c7d-dab1-41fa-9615-d5b92b9d3c6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:11.935702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2269c7d-dab1-41fa-9615-d5b92b9d3c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:11.935982 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2269c7d-dab1-41fa-9615-d5b92b9d3c6e admin admin] PATH_INFO: `/v3/users/1d839135353f4856b54758b3ece46178` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:11.937214 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2269c7d-dab1-41fa-9615-d5b92b9d3c6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d839135353f4856b54758b3ece46178)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:11.939219 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2269c7d-dab1-41fa-9615-d5b92b9d3c6e admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:11.990409 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e2269c7d-dab1-41fa-9615-d5b92b9d3c6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d839135353f4856b54758b3ece46178'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:11.991203 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e2269c7d-dab1-41fa-9615-d5b92b9d3c6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d839135353f4856b54758b3ece46178'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:12.012661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e2269c7d-dab1-41fa-9615-d5b92b9d3c6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d839135353f4856b54758b3ece46178'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:12.013522 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e2269c7d-dab1-41fa-9615-d5b92b9d3c6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d839135353f4856b54758b3ece46178'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:12.021513 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e2269c7d-dab1-41fa-9615-d5b92b9d3c6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d839135353f4856b54758b3ece46178'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:12.037790 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1138/2272] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:45:11 2022] DELETE /identity/v3/users/1d839135353f4856b54758b3ece46178 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:12.042433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdb324-c49a-496e-bb3c-d2a580ad428e None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:12.042788 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdb324-c49a-496e-bb3c-d2a580ad428e None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:12.043244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdb324-c49a-496e-bb3c-d2a580ad428e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:12.090966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-efcdb324-c49a-496e-bb3c-d2a580ad428e None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:12.092440 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-efcdb324-c49a-496e-bb3c-d2a580ad428e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:12.170556 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1135/2273] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:12.181585 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2269c7d-dab1-41fa-9615-d5b92b9d3c6e admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:12.183129 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e2269c7d-dab1-41fa-9615-d5b92b9d3c6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:12.253389 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2269c7d-dab1-41fa-9615-d5b92b9d3c6e admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:12.258842 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be9cb6b-0a3f-4706-8788-f6b225b0f99d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:12.259556 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9cb6b-0a3f-4706-8788-f6b225b0f99d service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:12.259968 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9cb6b-0a3f-4706-8788-f6b225b0f99d service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:12.260272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9cb6b-0a3f-4706-8788-f6b225b0f99d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:12.272532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be9cb6b-0a3f-4706-8788-f6b225b0f99d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:12.273675 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be9cb6b-0a3f-4706-8788-f6b225b0f99d service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:12.280127 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:12.291200 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1139/2274] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:45:12.989597 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcdb324-c49a-496e-bb3c-d2a580ad428e None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:12.992293 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-efcdb324-c49a-496e-bb3c-d2a580ad428e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:13.015945 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcdb324-c49a-496e-bb3c-d2a580ad428e None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:13.031195 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a458d8-a615-4179-bd35-373fc750808c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:13.032308 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a458d8-a615-4179-bd35-373fc750808c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:13.032782 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a458d8-a615-4179-bd35-373fc750808c admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:13.033240 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a458d8-a615-4179-bd35-373fc750808c admin admin] PATH_INFO: `/v3/projects/d1f0624454af4e53821b60c4ffe41d2d` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:13.036567 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a458d8-a615-4179-bd35-373fc750808c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1f0624454af4e53821b60c4ffe41d2d)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:13.039867 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a458d8-a615-4179-bd35-373fc750808c admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:13.127548 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a0a458d8-a615-4179-bd35-373fc750808c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1f0624454af4e53821b60c4ffe41d2d', 'request_id': 'req-a0a458d8-a615-4179-bd35-373fc750808c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:13.136755 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a0a458d8-a615-4179-bd35-373fc750808c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1f0624454af4e53821b60c4ffe41d2d', 'request_id': 'req-a0a458d8-a615-4179-bd35-373fc750808c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:13.144678 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a0a458d8-a615-4179-bd35-373fc750808c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1f0624454af4e53821b60c4ffe41d2d', 'request_id': 'req-a0a458d8-a615-4179-bd35-373fc750808c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:13.157356 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a0a458d8-a615-4179-bd35-373fc750808c admin admin] The token cache is being invalidate because project d1f0624454af4e53821b60c4ffe41d2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:45:13.157969 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a0a458d8-a615-4179-bd35-373fc750808c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:13.158899 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a0a458d8-a615-4179-bd35-373fc750808c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:13.160543 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1136/2275] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:45:12 2022] DELETE /identity/v3/projects/d1f0624454af4e53821b60c4ffe41d2d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:13.164621 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be9cb6b-0a3f-4706-8788-f6b225b0f99d service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:13.175807 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be9cb6b-0a3f-4706-8788-f6b225b0f99d service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:13.240081 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc576f-c6ee-48ae-ba63-fbad5d295bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:13.240874 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc576f-c6ee-48ae-ba63-fbad5d295bfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:13.241157 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc576f-c6ee-48ae-ba63-fbad5d295bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:13.241433 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc576f-c6ee-48ae-ba63-fbad5d295bfc admin admin] PATH_INFO: `/v3/users/adcb5bfc291a4eefb9b4fff6f140c782` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:13.242678 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc576f-c6ee-48ae-ba63-fbad5d295bfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=adcb5bfc291a4eefb9b4fff6f140c782)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:13.244668 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc576f-c6ee-48ae-ba63-fbad5d295bfc admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:13.306116 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e2dc576f-c6ee-48ae-ba63-fbad5d295bfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adcb5bfc291a4eefb9b4fff6f140c782'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:13.307058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e2dc576f-c6ee-48ae-ba63-fbad5d295bfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adcb5bfc291a4eefb9b4fff6f140c782'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:13.330623 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e2dc576f-c6ee-48ae-ba63-fbad5d295bfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adcb5bfc291a4eefb9b4fff6f140c782'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:13.331432 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e2dc576f-c6ee-48ae-ba63-fbad5d295bfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adcb5bfc291a4eefb9b4fff6f140c782'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:13.339457 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e2dc576f-c6ee-48ae-ba63-fbad5d295bfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adcb5bfc291a4eefb9b4fff6f140c782'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:13.351795 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1140/2276] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:45:13 2022] DELETE /identity/v3/users/adcb5bfc291a4eefb9b4fff6f140c782 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:45:13.924090 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a458d8-a615-4179-bd35-373fc750808c admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:13.927476 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-a0a458d8-a615-4179-bd35-373fc750808c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:13.953285 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a458d8-a615-4179-bd35-373fc750808c admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:14.036290 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b10c33-8fd5-4185-aeda-d96901a99a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:14.037384 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b10c33-8fd5-4185-aeda-d96901a99a9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:14.037551 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b10c33-8fd5-4185-aeda-d96901a99a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:14.037816 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b10c33-8fd5-4185-aeda-d96901a99a9a admin admin] PATH_INFO: `/v3/projects/2d19e07ec6d541b3a55064f898424d71` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:14.039341 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b10c33-8fd5-4185-aeda-d96901a99a9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d19e07ec6d541b3a55064f898424d71)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:14.041274 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b10c33-8fd5-4185-aeda-d96901a99a9a admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:14.044771 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:14.135258 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e4b10c33-8fd5-4185-aeda-d96901a99a9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d19e07ec6d541b3a55064f898424d71', 'request_id': 'req-e4b10c33-8fd5-4185-aeda-d96901a99a9a'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:14.145720 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e4b10c33-8fd5-4185-aeda-d96901a99a9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d19e07ec6d541b3a55064f898424d71', 'request_id': 'req-e4b10c33-8fd5-4185-aeda-d96901a99a9a'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:14.157955 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e4b10c33-8fd5-4185-aeda-d96901a99a9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d19e07ec6d541b3a55064f898424d71', 'request_id': 'req-e4b10c33-8fd5-4185-aeda-d96901a99a9a'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:14.179982 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e4b10c33-8fd5-4185-aeda-d96901a99a9a admin admin] The token cache is being invalidate because project 2d19e07ec6d541b3a55064f898424d71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:45:14.180559 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e4b10c33-8fd5-4185-aeda-d96901a99a9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:14.181554 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e4b10c33-8fd5-4185-aeda-d96901a99a9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:14.185251 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1137/2277] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:45:13 2022] DELETE /identity/v3/projects/2d19e07ec6d541b3a55064f898424d71 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:14.190479 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f0750-ecc3-42cd-b7f4-a5b193f2864e None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:14.190479 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f0750-ecc3-42cd-b7f4-a5b193f2864e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:14.191101 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f0750-ecc3-42cd-b7f4-a5b193f2864e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:14.230423 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-839f0750-ecc3-42cd-b7f4-a5b193f2864e None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:14.235509 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-839f0750-ecc3-42cd-b7f4-a5b193f2864e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:14.334348 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1141/2278] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:14.341995 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90105048-f0ea-4e69-a728-727dec48aac0 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:14.342245 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90105048-f0ea-4e69-a728-727dec48aac0 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:14.342532 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90105048-f0ea-4e69-a728-727dec48aac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:14.377388 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-90105048-f0ea-4e69-a728-727dec48aac0 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:14.388057 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-90105048-f0ea-4e69-a728-727dec48aac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:14.403789 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1138/2279] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:14.408189 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839f0750-ecc3-42cd-b7f4-a5b193f2864e None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:14.428923 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839f0750-ecc3-42cd-b7f4-a5b193f2864e None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:14.436067 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1f886-f23c-4600-970f-ebe40eba0b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:14.436434 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1f886-f23c-4600-970f-ebe40eba0b19 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:14.436724 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1f886-f23c-4600-970f-ebe40eba0b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:14.437024 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1f886-f23c-4600-970f-ebe40eba0b19 admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:14.437792 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e1f886-f23c-4600-970f-ebe40eba0b19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:45:14.438349 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e1f886-f23c-4600-970f-ebe40eba0b19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:14.440229 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e1f886-f23c-4600-970f-ebe40eba0b19 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:14.457413 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1142/2280] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:45:14.459177 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90105048-f0ea-4e69-a728-727dec48aac0 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:14.468589 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90105048-f0ea-4e69-a728-727dec48aac0 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:14.474479 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc9ef18-41ea-4f9f-82e4-be01d6905280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:14.475621 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc9ef18-41ea-4f9f-82e4-be01d6905280 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:14.475921 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc9ef18-41ea-4f9f-82e4-be01d6905280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:14.476251 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc9ef18-41ea-4f9f-82e4-be01d6905280 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:14.477038 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc9ef18-41ea-4f9f-82e4-be01d6905280 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:14.478864 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc9ef18-41ea-4f9f-82e4-be01d6905280 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:14.536120 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1139/2281] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:45:14 2022] POST /identity/v3/projects => generated 399 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:45:14.540742 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1f886-f23c-4600-970f-ebe40eba0b19 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:14.572762 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1f886-f23c-4600-970f-ebe40eba0b19 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:14.582811 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:14.664722 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e6d681-5130-4e0f-84b5-c463aa25ba1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:14.665502 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e6d681-5130-4e0f-84b5-c463aa25ba1d admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:14.666007 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e6d681-5130-4e0f-84b5-c463aa25ba1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:14.666191 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e6d681-5130-4e0f-84b5-c463aa25ba1d admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:14.667053 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e6d681-5130-4e0f-84b5-c463aa25ba1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:14.669071 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e6d681-5130-4e0f-84b5-c463aa25ba1d admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:14.691398 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1143/2282] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:45:14 2022] POST /identity/v3/users => generated 300 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:45:14.696139 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc9ef18-41ea-4f9f-82e4-be01d6905280 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:14.712339 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc9ef18-41ea-4f9f-82e4-be01d6905280 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:14.717308 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a787587b-d11c-4b34-a6ee-02b3f30a4a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:14.717903 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a787587b-d11c-4b34-a6ee-02b3f30a4a32 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:14.718191 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a787587b-d11c-4b34-a6ee-02b3f30a4a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:14.718471 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a787587b-d11c-4b34-a6ee-02b3f30a4a32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:14.719249 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a787587b-d11c-4b34-a6ee-02b3f30a4a32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:14.721272 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a787587b-d11c-4b34-a6ee-02b3f30a4a32 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:14.740302 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1140/2283] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:14.745850 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e6d681-5130-4e0f-84b5-c463aa25ba1d admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:14.759419 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e6d681-5130-4e0f-84b5-c463aa25ba1d admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:14.766265 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97b0c9-5030-456e-80e7-ccba9ec4fbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:14.767646 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97b0c9-5030-456e-80e7-ccba9ec4fbf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:14.767646 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97b0c9-5030-456e-80e7-ccba9ec4fbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:14.767881 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97b0c9-5030-456e-80e7-ccba9ec4fbf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:14.768722 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a97b0c9-5030-456e-80e7-ccba9ec4fbf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:14.770446 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a97b0c9-5030-456e-80e7-ccba9ec4fbf2 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:14.785747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1144/2284] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:45:14.789938 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a787587b-d11c-4b34-a6ee-02b3f30a4a32 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:14.802954 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a787587b-d11c-4b34-a6ee-02b3f30a4a32 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:14.807939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c7bac-d0a0-45d8-b34f-7b0f891eb8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:14.808925 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c7bac-d0a0-45d8-b34f-7b0f891eb8b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:14.809618 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c7bac-d0a0-45d8-b34f-7b0f891eb8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:14.809734 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c7bac-d0a0-45d8-b34f-7b0f891eb8b0 admin admin] PATH_INFO: `/v3/projects/a5e0016e11c442babd8faaf9b945a570/users/e1b5cb970d964724a0d1c298d3781b25/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:14.828392 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c7bac-d0a0-45d8-b34f-7b0f891eb8b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5e0016e11c442babd8faaf9b945a570, user_id=e1b5cb970d964724a0d1c298d3781b25, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:14.830676 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c7bac-d0a0-45d8-b34f-7b0f891eb8b0 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:14.847628 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1141/2285] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:45:14 2022] PUT /identity/v3/projects/a5e0016e11c442babd8faaf9b945a570/users/e1b5cb970d964724a0d1c298d3781b25/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:14.859061 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7d6f7-5987-4e56-844a-3881b99aed52 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:14.859289 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7d6f7-5987-4e56-844a-3881b99aed52 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:14.859563 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7d6f7-5987-4e56-844a-3881b99aed52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:14.890925 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-48a7d6f7-5987-4e56-844a-3881b99aed52 None None] MFA Rules not processed for user `e1b5cb970d964724a0d1c298d3781b25`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:14.892300 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-48a7d6f7-5987-4e56-844a-3881b99aed52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:15.008522 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1145/2286] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:15.012612 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c7bac-d0a0-45d8-b34f-7b0f891eb8b0 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:15.089604 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c7bac-d0a0-45d8-b34f-7b0f891eb8b0 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:15.094931 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de9a91-9616-4854-b566-b6f6a1ad05a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:15.095653 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de9a91-9616-4854-b566-b6f6a1ad05a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:15.095940 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de9a91-9616-4854-b566-b6f6a1ad05a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:15.096219 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de9a91-9616-4854-b566-b6f6a1ad05a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:15.096939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de9a91-9616-4854-b566-b6f6a1ad05a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:15.099344 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de9a91-9616-4854-b566-b6f6a1ad05a6 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:15.136019 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1142/2287] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:45:15 2022] POST /identity/v3/projects => generated 399 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:45:15.137522 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7d6f7-5987-4e56-844a-3881b99aed52 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:15.149276 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7d6f7-5987-4e56-844a-3881b99aed52 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:15.228974 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555034b1-5b64-42aa-8dff-321943db37bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:15.229778 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555034b1-5b64-42aa-8dff-321943db37bf admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:15.230190 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555034b1-5b64-42aa-8dff-321943db37bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:15.230561 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555034b1-5b64-42aa-8dff-321943db37bf admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:15.231399 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555034b1-5b64-42aa-8dff-321943db37bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:15.233475 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555034b1-5b64-42aa-8dff-321943db37bf admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:15.283809 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1146/2288] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:45:15 2022] POST /identity/v3/users => generated 306 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:45:15.288053 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de9a91-9616-4854-b566-b6f6a1ad05a6 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:15.299814 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de9a91-9616-4854-b566-b6f6a1ad05a6 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:15.305417 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f44103-f2c8-451e-952f-24d3ae935904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:15.306134 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f44103-f2c8-451e-952f-24d3ae935904 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:15.306486 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f44103-f2c8-451e-952f-24d3ae935904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:15.306701 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f44103-f2c8-451e-952f-24d3ae935904 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:15.307486 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f44103-f2c8-451e-952f-24d3ae935904 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:15.309124 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f44103-f2c8-451e-952f-24d3ae935904 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:15.330429 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1143/2289] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:15.331999 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555034b1-5b64-42aa-8dff-321943db37bf admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:15.342182 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:15.347727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555034b1-5b64-42aa-8dff-321943db37bf admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:15.356168 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aeb3ce-8196-4133-9b5c-b5558c6bc05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:15.357308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aeb3ce-8196-4133-9b5c-b5558c6bc05c admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:15.357676 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aeb3ce-8196-4133-9b5c-b5558c6bc05c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:15.358436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aeb3ce-8196-4133-9b5c-b5558c6bc05c admin admin] PATH_INFO: `/v3/projects/e0e4df6e9cab4f6ea5f9039ae73e4038/users/2dc972a5a7e44ca08bcbb00c25a1af88/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:15.389620 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aeb3ce-8196-4133-9b5c-b5558c6bc05c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0e4df6e9cab4f6ea5f9039ae73e4038, user_id=2dc972a5a7e44ca08bcbb00c25a1af88, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:15.392604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aeb3ce-8196-4133-9b5c-b5558c6bc05c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:15.417938 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1147/2290] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:45:15 2022] PUT /identity/v3/projects/e0e4df6e9cab4f6ea5f9039ae73e4038/users/2dc972a5a7e44ca08bcbb00c25a1af88/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:15.421945 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f44103-f2c8-451e-952f-24d3ae935904 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:15.430125 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:15.436035 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f44103-f2c8-451e-952f-24d3ae935904 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:15.515334 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b40271-11db-4ab1-8146-cc347faa41f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:15.516242 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b40271-11db-4ab1-8146-cc347faa41f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:15.516643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b40271-11db-4ab1-8146-cc347faa41f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:15.516984 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b40271-11db-4ab1-8146-cc347faa41f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:15.517789 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b40271-11db-4ab1-8146-cc347faa41f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:15.519680 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b40271-11db-4ab1-8146-cc347faa41f4 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:15.532146 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1144/2291] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:15.536280 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aeb3ce-8196-4133-9b5c-b5558c6bc05c admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:15.549713 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aeb3ce-8196-4133-9b5c-b5558c6bc05c admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:15.561159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975994e6-0876-449b-80fe-e0015cbbcbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:15.562367 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975994e6-0876-449b-80fe-e0015cbbcbe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:15.562865 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975994e6-0876-449b-80fe-e0015cbbcbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:15.563276 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975994e6-0876-449b-80fe-e0015cbbcbe5 admin admin] PATH_INFO: `/v3/projects/e0e4df6e9cab4f6ea5f9039ae73e4038/users/2dc972a5a7e44ca08bcbb00c25a1af88/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:15.566800 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975994e6-0876-449b-80fe-e0015cbbcbe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0e4df6e9cab4f6ea5f9039ae73e4038, user_id=2dc972a5a7e44ca08bcbb00c25a1af88, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:15.572119 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975994e6-0876-449b-80fe-e0015cbbcbe5 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:15.597355 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1148/2292] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:45:15 2022] PUT /identity/v3/projects/e0e4df6e9cab4f6ea5f9039ae73e4038/users/2dc972a5a7e44ca08bcbb00c25a1af88/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:45:15.608068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34543802-b87a-4845-9515-6cb50372f6a5 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:15.608434 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34543802-b87a-4845-9515-6cb50372f6a5 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:15.608809 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34543802-b87a-4845-9515-6cb50372f6a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:15.647119 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-34543802-b87a-4845-9515-6cb50372f6a5 None None] MFA Rules not processed for user `2dc972a5a7e44ca08bcbb00c25a1af88`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:15.649111 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-34543802-b87a-4845-9515-6cb50372f6a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:15.787468 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1145/2293] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:15 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:15.798386 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975994e6-0876-449b-80fe-e0015cbbcbe5 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:15.801492 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-975994e6-0876-449b-80fe-e0015cbbcbe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:15.919359 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975994e6-0876-449b-80fe-e0015cbbcbe5 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:15.923767 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8f3be9-7a70-4969-b20e-5b974616c925 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:15.924532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f3be9-7a70-4969-b20e-5b974616c925 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:15.925160 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f3be9-7a70-4969-b20e-5b974616c925 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:15.925522 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f3be9-7a70-4969-b20e-5b974616c925 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:15.936194 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8f3be9-7a70-4969-b20e-5b974616c925 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:15.943273 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8f3be9-7a70-4969-b20e-5b974616c925 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:15.956241 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1149/2294] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:15 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:45:16.065735 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34543802-b87a-4845-9515-6cb50372f6a5 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:16.077117 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34543802-b87a-4845-9515-6cb50372f6a5 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:16.091907 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c18e9-031a-4e9b-a835-2ce9f6260502 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:16.093068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c18e9-031a-4e9b-a835-2ce9f6260502 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:16.093562 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c18e9-031a-4e9b-a835-2ce9f6260502 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:16.094216 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c18e9-031a-4e9b-a835-2ce9f6260502 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:16.205840 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c18e9-031a-4e9b-a835-2ce9f6260502 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:16.207857 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c18e9-031a-4e9b-a835-2ce9f6260502 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:16.222152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868c5a9-724e-4602-b2ab-4430c544efda None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:16.222435 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868c5a9-724e-4602-b2ab-4430c544efda None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:16.222711 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868c5a9-724e-4602-b2ab-4430c544efda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:16.238891 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1146/2295] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:16 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:45:16.258543 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-e868c5a9-724e-4602-b2ab-4430c544efda None None] MFA Rules not processed for user `5d47f5d89bbb4d43844b1fb0794a3e69`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:16.259954 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e868c5a9-724e-4602-b2ab-4430c544efda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:16.338921 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1150/2296] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:45:16.342847 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c18e9-031a-4e9b-a835-2ce9f6260502 service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:16.344980 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-081c18e9-031a-4e9b-a835-2ce9f6260502 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:16.385322 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:16.438456 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c18e9-031a-4e9b-a835-2ce9f6260502 service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:16.446462 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196f6d1-3a21-4da2-bd97-b690f276a915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:16.447933 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196f6d1-3a21-4da2-bd97-b690f276a915 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:16.448301 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196f6d1-3a21-4da2-bd97-b690f276a915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:16.448576 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196f6d1-3a21-4da2-bd97-b690f276a915 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:16.449286 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f196f6d1-3a21-4da2-bd97-b690f276a915 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:16.452220 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f196f6d1-3a21-4da2-bd97-b690f276a915 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:16.498471 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1147/2297] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:45:16 2022] POST /identity/v3/projects => generated 373 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:45:16.503005 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e868c5a9-724e-4602-b2ab-4430c544efda None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:16.516604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e868c5a9-724e-4602-b2ab-4430c544efda None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:16.570817 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:16.638758 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ca7d56-34dc-4bd0-8e3e-05dd32c2e175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:16.639491 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca7d56-34dc-4bd0-8e3e-05dd32c2e175 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:16.639778 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca7d56-34dc-4bd0-8e3e-05dd32c2e175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:16.640056 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca7d56-34dc-4bd0-8e3e-05dd32c2e175 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:16.640762 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ca7d56-34dc-4bd0-8e3e-05dd32c2e175 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:16.642740 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ca7d56-34dc-4bd0-8e3e-05dd32c2e175 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:16.675075 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1151/2298] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:45:16 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:45:16.679475 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196f6d1-3a21-4da2-bd97-b690f276a915 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:16.692277 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196f6d1-3a21-4da2-bd97-b690f276a915 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:16.698241 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592289c-403e-4901-8978-24a373bc62fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:16.699099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592289c-403e-4901-8978-24a373bc62fc admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:16.699457 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592289c-403e-4901-8978-24a373bc62fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:16.699743 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592289c-403e-4901-8978-24a373bc62fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:16.700486 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a592289c-403e-4901-8978-24a373bc62fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:16.702100 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a592289c-403e-4901-8978-24a373bc62fc admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:16.720297 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1148/2299] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:16.721304 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ca7d56-34dc-4bd0-8e3e-05dd32c2e175 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:16.737235 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ca7d56-34dc-4bd0-8e3e-05dd32c2e175 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:16.744083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5dfaf-5e69-41e8-933a-0924cd50fd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:16.745093 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5dfaf-5e69-41e8-933a-0924cd50fd69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:16.745483 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5dfaf-5e69-41e8-933a-0924cd50fd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:16.745882 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5dfaf-5e69-41e8-933a-0924cd50fd69 admin admin] PATH_INFO: `/v3/projects/f5189f66322f4fb6a6e85bfdfca8b4e2/users/0d0af5aca957486ea840018795e438c5/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:16.764428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b5dfaf-5e69-41e8-933a-0924cd50fd69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5189f66322f4fb6a6e85bfdfca8b4e2, user_id=0d0af5aca957486ea840018795e438c5, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:16.767447 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b5dfaf-5e69-41e8-933a-0924cd50fd69 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:16.788023 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1152/2300] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:45:16 2022] PUT /identity/v3/projects/f5189f66322f4fb6a6e85bfdfca8b4e2/users/0d0af5aca957486ea840018795e438c5/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:45:16.791910 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592289c-403e-4901-8978-24a373bc62fc admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:16.805013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592289c-403e-4901-8978-24a373bc62fc admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:16.864401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc498e-d4c5-431f-8ce5-80fcb2ad1267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:16.866544 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc498e-d4c5-431f-8ce5-80fcb2ad1267 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:16.866544 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc498e-d4c5-431f-8ce5-80fcb2ad1267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:16.866544 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc498e-d4c5-431f-8ce5-80fcb2ad1267 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:16.867066 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc498e-d4c5-431f-8ce5-80fcb2ad1267 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:16.868233 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc498e-d4c5-431f-8ce5-80fcb2ad1267 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:16.880894 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1149/2301] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:16.889022 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5dfaf-5e69-41e8-933a-0924cd50fd69 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:16.900780 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5dfaf-5e69-41e8-933a-0924cd50fd69 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:16.906464 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12806187-8836-42ce-988e-34620319113a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:16.908015 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12806187-8836-42ce-988e-34620319113a admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:16.908015 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12806187-8836-42ce-988e-34620319113a admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:16.908015 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12806187-8836-42ce-988e-34620319113a admin admin] PATH_INFO: `/v3/projects/f5189f66322f4fb6a6e85bfdfca8b4e2/users/0d0af5aca957486ea840018795e438c5/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:16.910092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12806187-8836-42ce-988e-34620319113a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5189f66322f4fb6a6e85bfdfca8b4e2, user_id=0d0af5aca957486ea840018795e438c5, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:16.912897 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12806187-8836-42ce-988e-34620319113a admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:16.923879 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1153/2302] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:45:16 2022] PUT /identity/v3/projects/f5189f66322f4fb6a6e85bfdfca8b4e2/users/0d0af5aca957486ea840018795e438c5/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:22.086294 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc498e-d4c5-431f-8ce5-80fcb2ad1267 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:22.117671 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc498e-d4c5-431f-8ce5-80fcb2ad1267 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:22.225983 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b31dd-5804-4425-8927-d16a53abf1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:22.226892 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b31dd-5804-4425-8927-d16a53abf1c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:22.227216 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b31dd-5804-4425-8927-d16a53abf1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:22.227497 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b31dd-5804-4425-8927-d16a53abf1c9 admin admin] PATH_INFO: `/v3/users/e1b5cb970d964724a0d1c298d3781b25` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:22.231247 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b31dd-5804-4425-8927-d16a53abf1c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1b5cb970d964724a0d1c298d3781b25)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:22.233757 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b31dd-5804-4425-8927-d16a53abf1c9 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:22.298579 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:22.311229 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-005b31dd-5804-4425-8927-d16a53abf1c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1b5cb970d964724a0d1c298d3781b25'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:22.311633 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-005b31dd-5804-4425-8927-d16a53abf1c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1b5cb970d964724a0d1c298d3781b25'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:22.329929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-005b31dd-5804-4425-8927-d16a53abf1c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1b5cb970d964724a0d1c298d3781b25'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:22.341113 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-005b31dd-5804-4425-8927-d16a53abf1c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1b5cb970d964724a0d1c298d3781b25'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:22.355541 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-005b31dd-5804-4425-8927-d16a53abf1c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1b5cb970d964724a0d1c298d3781b25'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:22.363008 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1150/2303] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:45:22 2022] DELETE /identity/v3/users/e1b5cb970d964724a0d1c298d3781b25 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:22.367297 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756b52e-9bf7-4de3-b7d1-c5c1f74c7e67 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:22.367796 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756b52e-9bf7-4de3-b7d1-c5c1f74c7e67 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:22.367877 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756b52e-9bf7-4de3-b7d1-c5c1f74c7e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:22.392220 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-a756b52e-9bf7-4de3-b7d1-c5c1f74c7e67 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:22.394031 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a756b52e-9bf7-4de3-b7d1-c5c1f74c7e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:22.486515 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1154/2304] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:22.493953 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b31dd-5804-4425-8927-d16a53abf1c9 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:22.495446 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-005b31dd-5804-4425-8927-d16a53abf1c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:22.589752 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b31dd-5804-4425-8927-d16a53abf1c9 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:22.594905 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ccc0e-528b-43b0-a54f-ee3e48a189ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:22.595623 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ccc0e-528b-43b0-a54f-ee3e48a189ec service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:22.595912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ccc0e-528b-43b0-a54f-ee3e48a189ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:22.596191 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ccc0e-528b-43b0-a54f-ee3e48a189ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:22.609408 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ccc0e-528b-43b0-a54f-ee3e48a189ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:22.609673 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ccc0e-528b-43b0-a54f-ee3e48a189ec service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:22.629852 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1151/2305] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:45:22.743470 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b51105-59a6-41cb-bc56-218db0d9937a None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:22.743793 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b51105-59a6-41cb-bc56-218db0d9937a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:22.744171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b51105-59a6-41cb-bc56-218db0d9937a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:22.778212 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-e0b51105-59a6-41cb-bc56-218db0d9937a None None] MFA Rules not processed for user `0d0af5aca957486ea840018795e438c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:22.780802 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e0b51105-59a6-41cb-bc56-218db0d9937a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:22.902305 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1155/2306] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:22 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:22.912463 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ccc0e-528b-43b0-a54f-ee3e48a189ec service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:22.914183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-297ccc0e-528b-43b0-a54f-ee3e48a189ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:22.997746 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b51105-59a6-41cb-bc56-218db0d9937a None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:23.000185 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e0b51105-59a6-41cb-bc56-218db0d9937a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:23.022675 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b51105-59a6-41cb-bc56-218db0d9937a None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:23.029276 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a4772-e87b-4e6e-8d1c-1b2d24345eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:23.030248 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a4772-e87b-4e6e-8d1c-1b2d24345eb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:23.030824 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a4772-e87b-4e6e-8d1c-1b2d24345eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:23.030824 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a4772-e87b-4e6e-8d1c-1b2d24345eb2 admin admin] PATH_INFO: `/v3/projects/a5e0016e11c442babd8faaf9b945a570` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:23.033094 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328a4772-e87b-4e6e-8d1c-1b2d24345eb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5e0016e11c442babd8faaf9b945a570)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:23.035808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328a4772-e87b-4e6e-8d1c-1b2d24345eb2 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:23.058454 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:23.076748 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ccc0e-528b-43b0-a54f-ee3e48a189ec service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:23.083733 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54608a07-7a37-444b-9433-cbc18af95c37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:23.084661 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54608a07-7a37-444b-9433-cbc18af95c37 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:23.085639 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54608a07-7a37-444b-9433-cbc18af95c37 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:23.086681 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54608a07-7a37-444b-9433-cbc18af95c37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:23.092323 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-54608a07-7a37-444b-9433-cbc18af95c37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:23.107497 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-328a4772-e87b-4e6e-8d1c-1b2d24345eb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5e0016e11c442babd8faaf9b945a570', 'request_id': 'req-328a4772-e87b-4e6e-8d1c-1b2d24345eb2'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:23.116198 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-328a4772-e87b-4e6e-8d1c-1b2d24345eb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5e0016e11c442babd8faaf9b945a570', 'request_id': 'req-328a4772-e87b-4e6e-8d1c-1b2d24345eb2'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:23.123947 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-328a4772-e87b-4e6e-8d1c-1b2d24345eb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5e0016e11c442babd8faaf9b945a570', 'request_id': 'req-328a4772-e87b-4e6e-8d1c-1b2d24345eb2'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:23.132899 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:23.137073 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-328a4772-e87b-4e6e-8d1c-1b2d24345eb2 admin admin] The token cache is being invalidate because project a5e0016e11c442babd8faaf9b945a570 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:45:23.137355 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-328a4772-e87b-4e6e-8d1c-1b2d24345eb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:23.138026 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-328a4772-e87b-4e6e-8d1c-1b2d24345eb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:23.139736 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1156/2307] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:45:22 2022] DELETE /identity/v3/projects/a5e0016e11c442babd8faaf9b945a570 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:23.144031 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a4772-e87b-4e6e-8d1c-1b2d24345eb2 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:23.145464 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-328a4772-e87b-4e6e-8d1c-1b2d24345eb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:23.164282 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54608a07-7a37-444b-9433-cbc18af95c37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:23.165986 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54608a07-7a37-444b-9433-cbc18af95c37 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:23.175361 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1152/2308] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:45:23.213602 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a4772-e87b-4e6e-8d1c-1b2d24345eb2 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:23.219243 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47dd010-f7af-4d8e-b63d-989f827c094d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:23.220037 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47dd010-f7af-4d8e-b63d-989f827c094d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:23.220330 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47dd010-f7af-4d8e-b63d-989f827c094d admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:23.220637 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47dd010-f7af-4d8e-b63d-989f827c094d admin admin] PATH_INFO: `/v3/users/2dc972a5a7e44ca08bcbb00c25a1af88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:23.222145 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47dd010-f7af-4d8e-b63d-989f827c094d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dc972a5a7e44ca08bcbb00c25a1af88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:23.224319 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47dd010-f7af-4d8e-b63d-989f827c094d admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:23.300017 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b47dd010-f7af-4d8e-b63d-989f827c094d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dc972a5a7e44ca08bcbb00c25a1af88'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:23.300857 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b47dd010-f7af-4d8e-b63d-989f827c094d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dc972a5a7e44ca08bcbb00c25a1af88'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:23.332832 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b47dd010-f7af-4d8e-b63d-989f827c094d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dc972a5a7e44ca08bcbb00c25a1af88'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:23.333656 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b47dd010-f7af-4d8e-b63d-989f827c094d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dc972a5a7e44ca08bcbb00c25a1af88'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:23.343302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b47dd010-f7af-4d8e-b63d-989f827c094d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dc972a5a7e44ca08bcbb00c25a1af88'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:23.357360 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1157/2309] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:45:23 2022] DELETE /identity/v3/users/2dc972a5a7e44ca08bcbb00c25a1af88 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:45:23.700750 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54608a07-7a37-444b-9433-cbc18af95c37 service nova] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:23.725538 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54608a07-7a37-444b-9433-cbc18af95c37 service nova] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:23.822152 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfb2b4-50b2-4164-90c2-a60f30da2e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:23.823278 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfb2b4-50b2-4164-90c2-a60f30da2e2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:23.823617 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfb2b4-50b2-4164-90c2-a60f30da2e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:23.823968 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfb2b4-50b2-4164-90c2-a60f30da2e2c admin admin] PATH_INFO: `/v3/projects/e0e4df6e9cab4f6ea5f9039ae73e4038` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:23.825494 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfb2b4-50b2-4164-90c2-a60f30da2e2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0e4df6e9cab4f6ea5f9039ae73e4038)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:23.828175 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfb2b4-50b2-4164-90c2-a60f30da2e2c admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:23.905806 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-33dfb2b4-50b2-4164-90c2-a60f30da2e2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0e4df6e9cab4f6ea5f9039ae73e4038', 'request_id': 'req-33dfb2b4-50b2-4164-90c2-a60f30da2e2c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:23.915896 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-33dfb2b4-50b2-4164-90c2-a60f30da2e2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0e4df6e9cab4f6ea5f9039ae73e4038', 'request_id': 'req-33dfb2b4-50b2-4164-90c2-a60f30da2e2c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:23.923398 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-33dfb2b4-50b2-4164-90c2-a60f30da2e2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0e4df6e9cab4f6ea5f9039ae73e4038', 'request_id': 'req-33dfb2b4-50b2-4164-90c2-a60f30da2e2c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:23.940729 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-33dfb2b4-50b2-4164-90c2-a60f30da2e2c admin admin] The token cache is being invalidate because project e0e4df6e9cab4f6ea5f9039ae73e4038 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:45:23.941040 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-33dfb2b4-50b2-4164-90c2-a60f30da2e2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:23.941803 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-33dfb2b4-50b2-4164-90c2-a60f30da2e2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:23.947100 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1153/2310] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:45:23 2022] DELETE /identity/v3/projects/e0e4df6e9cab4f6ea5f9039ae73e4038 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:23.949860 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050cc61-3d6c-4348-837f-fc7af46428cd None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:23.950399 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050cc61-3d6c-4348-837f-fc7af46428cd None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:23.950399 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050cc61-3d6c-4348-837f-fc7af46428cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:23.977363 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-a050cc61-3d6c-4348-837f-fc7af46428cd None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:23.979527 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a050cc61-3d6c-4348-837f-fc7af46428cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:23.996119 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:24.067863 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1158/2311] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:24.078115 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b525de9-383b-4279-8bc4-66478e48d67b None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:24.078115 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b525de9-383b-4279-8bc4-66478e48d67b None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:24.078520 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b525de9-383b-4279-8bc4-66478e48d67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:24.113598 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-6b525de9-383b-4279-8bc4-66478e48d67b None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:24.116189 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-6b525de9-383b-4279-8bc4-66478e48d67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:24.129711 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1154/2312] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:24.134858 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a050cc61-3d6c-4348-837f-fc7af46428cd None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:24.150326 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a050cc61-3d6c-4348-837f-fc7af46428cd None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:24.158366 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5d4c9-98a9-4aff-89d5-58ac532a332a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:24.159072 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5d4c9-98a9-4aff-89d5-58ac532a332a admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:24.159362 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5d4c9-98a9-4aff-89d5-58ac532a332a admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:24.159640 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5d4c9-98a9-4aff-89d5-58ac532a332a admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:24.160388 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e5d4c9-98a9-4aff-89d5-58ac532a332a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:45:24.160699 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e5d4c9-98a9-4aff-89d5-58ac532a332a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:24.162954 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e5d4c9-98a9-4aff-89d5-58ac532a332a admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:24.179040 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1159/2313] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:45:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:45:24.183129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b525de9-383b-4279-8bc4-66478e48d67b None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:24.196017 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b525de9-383b-4279-8bc4-66478e48d67b None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:24.201069 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f71fb2-5e6f-4ce6-9357-ee19ff776e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:24.201881 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f71fb2-5e6f-4ce6-9357-ee19ff776e77 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:24.202173 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f71fb2-5e6f-4ce6-9357-ee19ff776e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:24.202453 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f71fb2-5e6f-4ce6-9357-ee19ff776e77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:24.203214 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f71fb2-5e6f-4ce6-9357-ee19ff776e77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:24.205154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f71fb2-5e6f-4ce6-9357-ee19ff776e77 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:24.238478 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1155/2314] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:45:24 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:45:24.242899 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5d4c9-98a9-4aff-89d5-58ac532a332a admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:24.253286 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5d4c9-98a9-4aff-89d5-58ac532a332a admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:24.372508 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b8f4fd-e990-4714-9497-f2b2f28801ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:24.375338 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8f4fd-e990-4714-9497-f2b2f28801ab admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:24.375838 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8f4fd-e990-4714-9497-f2b2f28801ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:24.376272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8f4fd-e990-4714-9497-f2b2f28801ab admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:24.377126 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b8f4fd-e990-4714-9497-f2b2f28801ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:24.380080 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b8f4fd-e990-4714-9497-f2b2f28801ab admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:24.418412 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1160/2315] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:45:24 2022] POST /identity/v3/users => generated 291 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:45:24.418669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f71fb2-5e6f-4ce6-9357-ee19ff776e77 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:24.431480 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f71fb2-5e6f-4ce6-9357-ee19ff776e77 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:24.441202 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7af8c-c99c-454c-8b18-c66b9e6e1b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:24.442073 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7af8c-c99c-454c-8b18-c66b9e6e1b5c admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:24.442459 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7af8c-c99c-454c-8b18-c66b9e6e1b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:24.443598 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7af8c-c99c-454c-8b18-c66b9e6e1b5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:24.445678 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd7af8c-c99c-454c-8b18-c66b9e6e1b5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:24.448329 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd7af8c-c99c-454c-8b18-c66b9e6e1b5c admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:24.464302 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1156/2316] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:24 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:45:24.468593 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b8f4fd-e990-4714-9497-f2b2f28801ab admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:24.481409 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b8f4fd-e990-4714-9497-f2b2f28801ab admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:24.487029 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef174f5-0202-46cd-8d16-c77347856eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:24.487944 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef174f5-0202-46cd-8d16-c77347856eb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:24.488550 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef174f5-0202-46cd-8d16-c77347856eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:24.489459 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef174f5-0202-46cd-8d16-c77347856eb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:24.491490 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef174f5-0202-46cd-8d16-c77347856eb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:24.494099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef174f5-0202-46cd-8d16-c77347856eb9 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:24.509572 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1161/2317] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:24 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:24.514102 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7af8c-c99c-454c-8b18-c66b9e6e1b5c admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:24.529446 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c9fff-09bf-48ed-b7e8-8812310e0b4a None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:24.529639 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c9fff-09bf-48ed-b7e8-8812310e0b4a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:24.529928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c9fff-09bf-48ed-b7e8-8812310e0b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:24.533558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7af8c-c99c-454c-8b18-c66b9e6e1b5c admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:24.544013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc110c46-e50d-4629-8009-eea6c86118b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:24.545215 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc110c46-e50d-4629-8009-eea6c86118b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:24.545661 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc110c46-e50d-4629-8009-eea6c86118b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:24.546206 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc110c46-e50d-4629-8009-eea6c86118b5 admin admin] PATH_INFO: `/v3/projects/5cc639fdc5d7471fa97303bcd53b2218/users/254414ae95ae425d9a97d8f966949dc2/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:24.570739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-409c9fff-09bf-48ed-b7e8-8812310e0b4a None None] MFA Rules not processed for user `5d47f5d89bbb4d43844b1fb0794a3e69`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:24.572096 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-409c9fff-09bf-48ed-b7e8-8812310e0b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:24.575253 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc110c46-e50d-4629-8009-eea6c86118b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cc639fdc5d7471fa97303bcd53b2218, user_id=254414ae95ae425d9a97d8f966949dc2, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:24.578613 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc110c46-e50d-4629-8009-eea6c86118b5 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:24.591874 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:24.603590 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1157/2318] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:45:24 2022] PUT /identity/v3/projects/5cc639fdc5d7471fa97303bcd53b2218/users/254414ae95ae425d9a97d8f966949dc2/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:24.615897 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16860c5e-b238-498c-a074-d2b00334e4a3 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:24.617936 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16860c5e-b238-498c-a074-d2b00334e4a3 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:24.619420 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16860c5e-b238-498c-a074-d2b00334e4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:24.652175 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1162/2319] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:24.655121 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-16860c5e-b238-498c-a074-d2b00334e4a3 None None] MFA Rules not processed for user `254414ae95ae425d9a97d8f966949dc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:24.656605 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-16860c5e-b238-498c-a074-d2b00334e4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:24.758730 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1158/2320] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:24.762830 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c9fff-09bf-48ed-b7e8-8812310e0b4a None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:24.777254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c9fff-09bf-48ed-b7e8-8812310e0b4a None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:24.817542 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:24.868116 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb471b9b-94e7-41a2-8fed-b50137e3ee1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:24.868796 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb471b9b-94e7-41a2-8fed-b50137e3ee1c admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:24.869085 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb471b9b-94e7-41a2-8fed-b50137e3ee1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:24.869348 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb471b9b-94e7-41a2-8fed-b50137e3ee1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:24.870047 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb471b9b-94e7-41a2-8fed-b50137e3ee1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:24.873075 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb471b9b-94e7-41a2-8fed-b50137e3ee1c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:24.910153 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1163/2321] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:45:24 2022] POST /identity/v3/projects => generated 381 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:45:24.912120 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16860c5e-b238-498c-a074-d2b00334e4a3 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:24.934156 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16860c5e-b238-498c-a074-d2b00334e4a3 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:25.018625 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9441ec6-795b-43bf-a1de-4002a63e1c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:25.022968 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9441ec6-795b-43bf-a1de-4002a63e1c73 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:25.022968 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9441ec6-795b-43bf-a1de-4002a63e1c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:25.023196 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9441ec6-795b-43bf-a1de-4002a63e1c73 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:25.024167 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9441ec6-795b-43bf-a1de-4002a63e1c73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:25.027828 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9441ec6-795b-43bf-a1de-4002a63e1c73 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:25.087170 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb471b9b-94e7-41a2-8fed-b50137e3ee1c admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:25.089545 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1159/2322] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:45:24 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:45:25.105519 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb471b9b-94e7-41a2-8fed-b50137e3ee1c admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:25.112588 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615407a5-7ba5-4d53-9502-02f6d82eef58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:25.113618 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615407a5-7ba5-4d53-9502-02f6d82eef58 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:25.113760 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615407a5-7ba5-4d53-9502-02f6d82eef58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:25.114164 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615407a5-7ba5-4d53-9502-02f6d82eef58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:25.115664 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615407a5-7ba5-4d53-9502-02f6d82eef58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:25.118571 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615407a5-7ba5-4d53-9502-02f6d82eef58 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:25.139472 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9441ec6-795b-43bf-a1de-4002a63e1c73 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:25.147471 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1164/2323] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:25.153239 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9441ec6-795b-43bf-a1de-4002a63e1c73 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:25.161096 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b103861e-86a2-4faf-9683-d30f3f54ccf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:25.162798 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103861e-86a2-4faf-9683-d30f3f54ccf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:25.163333 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103861e-86a2-4faf-9683-d30f3f54ccf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:25.163762 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103861e-86a2-4faf-9683-d30f3f54ccf3 admin admin] PATH_INFO: `/v3/projects/b774da908fee4b9b98e94be64bf3f056/users/71d691f4f7aa411982edf5d0c56be378/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:25.185765 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b103861e-86a2-4faf-9683-d30f3f54ccf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b774da908fee4b9b98e94be64bf3f056, user_id=71d691f4f7aa411982edf5d0c56be378, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:25.197635 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b103861e-86a2-4faf-9683-d30f3f54ccf3 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:25.236127 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615407a5-7ba5-4d53-9502-02f6d82eef58 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:25.237203 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1160/2324] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:45:25 2022] PUT /identity/v3/projects/b774da908fee4b9b98e94be64bf3f056/users/71d691f4f7aa411982edf5d0c56be378/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:25.257103 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615407a5-7ba5-4d53-9502-02f6d82eef58 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:25.406800 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeb9b69-91af-4c3c-adbf-d4111eba90a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:25.407557 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb9b69-91af-4c3c-adbf-d4111eba90a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:25.408070 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb9b69-91af-4c3c-adbf-d4111eba90a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:25.408348 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb9b69-91af-4c3c-adbf-d4111eba90a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:25.409107 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeb9b69-91af-4c3c-adbf-d4111eba90a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:25.411031 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeb9b69-91af-4c3c-adbf-d4111eba90a7 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:25.424169 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1165/2325] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:25.428241 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b103861e-86a2-4faf-9683-d30f3f54ccf3 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:25.440997 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b103861e-86a2-4faf-9683-d30f3f54ccf3 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:25.446233 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511b68b-6489-4fd4-ac3a-bee9b442f625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:25.447206 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511b68b-6489-4fd4-ac3a-bee9b442f625 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:25.447496 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511b68b-6489-4fd4-ac3a-bee9b442f625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:25.447967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511b68b-6489-4fd4-ac3a-bee9b442f625 admin admin] PATH_INFO: `/v3/projects/b774da908fee4b9b98e94be64bf3f056/users/71d691f4f7aa411982edf5d0c56be378/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:25.453511 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f511b68b-6489-4fd4-ac3a-bee9b442f625 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b774da908fee4b9b98e94be64bf3f056, user_id=71d691f4f7aa411982edf5d0c56be378, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:25.455641 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f511b68b-6489-4fd4-ac3a-bee9b442f625 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:25.478817 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1161/2326] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:45:25 2022] PUT /identity/v3/projects/b774da908fee4b9b98e94be64bf3f056/users/71d691f4f7aa411982edf5d0c56be378/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:25.489081 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae71e1e-5fa0-4207-9dae-988cfc66a8cb None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:25.489632 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae71e1e-5fa0-4207-9dae-988cfc66a8cb None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:25.489998 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae71e1e-5fa0-4207-9dae-988cfc66a8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:25.570706 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-cae71e1e-5fa0-4207-9dae-988cfc66a8cb None None] MFA Rules not processed for user `71d691f4f7aa411982edf5d0c56be378`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:25.572031 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-cae71e1e-5fa0-4207-9dae-988cfc66a8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:25.721274 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1166/2327] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:25 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:45:25.735937 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511b68b-6489-4fd4-ac3a-bee9b442f625 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:25.737753 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f511b68b-6489-4fd4-ac3a-bee9b442f625 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:25.804319 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:25.822499 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511b68b-6489-4fd4-ac3a-bee9b442f625 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:25.825831 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f4fd1-19cd-4790-9c7d-651ec2802b9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:25.826480 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f4fd1-19cd-4790-9c7d-651ec2802b9e service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:25.826761 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f4fd1-19cd-4790-9c7d-651ec2802b9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:25.827069 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f4fd1-19cd-4790-9c7d-651ec2802b9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:25.838639 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6f4fd1-19cd-4790-9c7d-651ec2802b9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:25.840123 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6f4fd1-19cd-4790-9c7d-651ec2802b9e service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:25.851496 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1162/2328] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:25 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:45:25.971446 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae71e1e-5fa0-4207-9dae-988cfc66a8cb None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:25.988517 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae71e1e-5fa0-4207-9dae-988cfc66a8cb None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:25.997698 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d959099-fda9-4875-8c7d-26e2d72d6462 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:25.998303 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d959099-fda9-4875-8c7d-26e2d72d6462 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:25.998889 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d959099-fda9-4875-8c7d-26e2d72d6462 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:25.999522 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d959099-fda9-4875-8c7d-26e2d72d6462 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:26.039191 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:26.090465 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d959099-fda9-4875-8c7d-26e2d72d6462 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:26.091983 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d959099-fda9-4875-8c7d-26e2d72d6462 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:26.103957 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1167/2329] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:45:26.478682 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f4fd1-19cd-4790-9c7d-651ec2802b9e service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:26.562730 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f4fd1-19cd-4790-9c7d-651ec2802b9e service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:26.569157 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11448b1d-025f-4863-aac4-e03e767a4ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:26.570097 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11448b1d-025f-4863-aac4-e03e767a4ca7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:26.570384 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11448b1d-025f-4863-aac4-e03e767a4ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:26.570659 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11448b1d-025f-4863-aac4-e03e767a4ca7 admin admin] PATH_INFO: `/v3/users/254414ae95ae425d9a97d8f966949dc2` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:26.571991 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11448b1d-025f-4863-aac4-e03e767a4ca7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=254414ae95ae425d9a97d8f966949dc2)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:26.573968 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11448b1d-025f-4863-aac4-e03e767a4ca7 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:26.634070 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-11448b1d-025f-4863-aac4-e03e767a4ca7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '254414ae95ae425d9a97d8f966949dc2'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:26.634857 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-11448b1d-025f-4863-aac4-e03e767a4ca7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '254414ae95ae425d9a97d8f966949dc2'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:26.654005 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-11448b1d-025f-4863-aac4-e03e767a4ca7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '254414ae95ae425d9a97d8f966949dc2'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:26.663979 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-11448b1d-025f-4863-aac4-e03e767a4ca7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '254414ae95ae425d9a97d8f966949dc2'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:26.676000 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-11448b1d-025f-4863-aac4-e03e767a4ca7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '254414ae95ae425d9a97d8f966949dc2'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:26.685331 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1163/2330] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:45:26 2022] DELETE /identity/v3/users/254414ae95ae425d9a97d8f966949dc2 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:26.689891 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a40ca2-468c-4b76-813a-518273ad6f55 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:26.690179 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a40ca2-468c-4b76-813a-518273ad6f55 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:26.690456 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a40ca2-468c-4b76-813a-518273ad6f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:26.710723 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-d5a40ca2-468c-4b76-813a-518273ad6f55 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:26.711981 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d5a40ca2-468c-4b76-813a-518273ad6f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:26.791552 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1168/2331] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:26.801518 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11448b1d-025f-4863-aac4-e03e767a4ca7 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:26.803153 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-11448b1d-025f-4863-aac4-e03e767a4ca7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:26.880202 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11448b1d-025f-4863-aac4-e03e767a4ca7 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:26.884033 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6049b-7833-4ab5-bd6a-581587cbfd70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:26.884680 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6049b-7833-4ab5-bd6a-581587cbfd70 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:26.884963 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6049b-7833-4ab5-bd6a-581587cbfd70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:26.885248 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6049b-7833-4ab5-bd6a-581587cbfd70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:26.896181 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e6049b-7833-4ab5-bd6a-581587cbfd70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:26.897633 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e6049b-7833-4ab5-bd6a-581587cbfd70 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:26.912020 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1164/2332] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:45:27.261660 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a40ca2-468c-4b76-813a-518273ad6f55 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:27.263420 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d5a40ca2-468c-4b76-813a-518273ad6f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:27.284233 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a40ca2-468c-4b76-813a-518273ad6f55 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:27.289931 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec03c21-7c1e-4879-920d-c9fdc7cff7e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:27.290939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec03c21-7c1e-4879-920d-c9fdc7cff7e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:27.291372 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec03c21-7c1e-4879-920d-c9fdc7cff7e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:27.291821 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec03c21-7c1e-4879-920d-c9fdc7cff7e0 admin admin] PATH_INFO: `/v3/projects/5cc639fdc5d7471fa97303bcd53b2218` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:27.293465 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec03c21-7c1e-4879-920d-c9fdc7cff7e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cc639fdc5d7471fa97303bcd53b2218)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:27.296000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec03c21-7c1e-4879-920d-c9fdc7cff7e0 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:27.357346 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-dec03c21-7c1e-4879-920d-c9fdc7cff7e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cc639fdc5d7471fa97303bcd53b2218', 'request_id': 'req-dec03c21-7c1e-4879-920d-c9fdc7cff7e0'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:27.370263 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-dec03c21-7c1e-4879-920d-c9fdc7cff7e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cc639fdc5d7471fa97303bcd53b2218', 'request_id': 'req-dec03c21-7c1e-4879-920d-c9fdc7cff7e0'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:27.380994 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-dec03c21-7c1e-4879-920d-c9fdc7cff7e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cc639fdc5d7471fa97303bcd53b2218', 'request_id': 'req-dec03c21-7c1e-4879-920d-c9fdc7cff7e0'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:27.393139 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-dec03c21-7c1e-4879-920d-c9fdc7cff7e0 admin admin] The token cache is being invalidate because project 5cc639fdc5d7471fa97303bcd53b2218 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:45:27.393437 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-dec03c21-7c1e-4879-920d-c9fdc7cff7e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:27.394135 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-dec03c21-7c1e-4879-920d-c9fdc7cff7e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:27.399098 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1169/2333] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:45:27 2022] DELETE /identity/v3/projects/5cc639fdc5d7471fa97303bcd53b2218 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:27.400693 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6049b-7833-4ab5-bd6a-581587cbfd70 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:27.407754 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:27.411641 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6049b-7833-4ab5-bd6a-581587cbfd70 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:27.474702 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c7d007-fc88-4548-9918-bc7aa36219d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:27.475907 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7d007-fc88-4548-9918-bc7aa36219d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:27.476229 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7d007-fc88-4548-9918-bc7aa36219d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:27.476507 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7d007-fc88-4548-9918-bc7aa36219d8 admin admin] PATH_INFO: `/v3/users/71d691f4f7aa411982edf5d0c56be378` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:27.478077 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c7d007-fc88-4548-9918-bc7aa36219d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71d691f4f7aa411982edf5d0c56be378)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:27.480508 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c7d007-fc88-4548-9918-bc7aa36219d8 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:27.536628 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-76c7d007-fc88-4548-9918-bc7aa36219d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71d691f4f7aa411982edf5d0c56be378'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:27.537429 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-76c7d007-fc88-4548-9918-bc7aa36219d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71d691f4f7aa411982edf5d0c56be378'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:27.551667 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-76c7d007-fc88-4548-9918-bc7aa36219d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71d691f4f7aa411982edf5d0c56be378'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:27.565313 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-76c7d007-fc88-4548-9918-bc7aa36219d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71d691f4f7aa411982edf5d0c56be378'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:27.581146 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-76c7d007-fc88-4548-9918-bc7aa36219d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71d691f4f7aa411982edf5d0c56be378'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:27.588692 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1165/2334] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:45:27 2022] DELETE /identity/v3/users/71d691f4f7aa411982edf5d0c56be378 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:27.900409 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec03c21-7c1e-4879-920d-c9fdc7cff7e0 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:27.902334 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-dec03c21-7c1e-4879-920d-c9fdc7cff7e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:27.920921 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec03c21-7c1e-4879-920d-c9fdc7cff7e0 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:27.974379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:27.990166 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8148c67b-5bcd-467c-a1db-5c63b472b99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:27.991325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148c67b-5bcd-467c-a1db-5c63b472b99a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:27.991636 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148c67b-5bcd-467c-a1db-5c63b472b99a admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:27.991996 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148c67b-5bcd-467c-a1db-5c63b472b99a admin admin] PATH_INFO: `/v3/projects/b774da908fee4b9b98e94be64bf3f056` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:27.993621 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8148c67b-5bcd-467c-a1db-5c63b472b99a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b774da908fee4b9b98e94be64bf3f056)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:27.996060 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8148c67b-5bcd-467c-a1db-5c63b472b99a admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:28.049182 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-8148c67b-5bcd-467c-a1db-5c63b472b99a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b774da908fee4b9b98e94be64bf3f056', 'request_id': 'req-8148c67b-5bcd-467c-a1db-5c63b472b99a'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:28.057203 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-8148c67b-5bcd-467c-a1db-5c63b472b99a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b774da908fee4b9b98e94be64bf3f056', 'request_id': 'req-8148c67b-5bcd-467c-a1db-5c63b472b99a'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:28.064077 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-8148c67b-5bcd-467c-a1db-5c63b472b99a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b774da908fee4b9b98e94be64bf3f056', 'request_id': 'req-8148c67b-5bcd-467c-a1db-5c63b472b99a'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:28.076647 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-8148c67b-5bcd-467c-a1db-5c63b472b99a admin admin] The token cache is being invalidate because project b774da908fee4b9b98e94be64bf3f056 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:45:28.076925 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-8148c67b-5bcd-467c-a1db-5c63b472b99a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:28.077634 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-8148c67b-5bcd-467c-a1db-5c63b472b99a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:28.079278 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1170/2335] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:45:27 2022] DELETE /identity/v3/projects/b774da908fee4b9b98e94be64bf3f056 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:28.084756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e256dcc-d5b0-489a-93d0-5101be3aeb0c None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:28.085052 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e256dcc-d5b0-489a-93d0-5101be3aeb0c None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:28.085371 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e256dcc-d5b0-489a-93d0-5101be3aeb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:28.104053 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-7e256dcc-d5b0-489a-93d0-5101be3aeb0c None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:28.105235 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-7e256dcc-d5b0-489a-93d0-5101be3aeb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:28.167519 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1166/2336] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:28.176533 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49175ee-cdea-411d-8d51-7fd3fb0548b4 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:28.177075 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49175ee-cdea-411d-8d51-7fd3fb0548b4 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:28.177658 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49175ee-cdea-411d-8d51-7fd3fb0548b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:28.199067 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-a49175ee-cdea-411d-8d51-7fd3fb0548b4 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:28.200305 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a49175ee-cdea-411d-8d51-7fd3fb0548b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:28.219847 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1171/2337] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:28.224501 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e256dcc-d5b0-489a-93d0-5101be3aeb0c None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:28.237343 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e256dcc-d5b0-489a-93d0-5101be3aeb0c None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:28.242179 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e5a08f-3da9-435b-82c9-7d95c09c098c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:28.242851 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5a08f-3da9-435b-82c9-7d95c09c098c admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:28.243228 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5a08f-3da9-435b-82c9-7d95c09c098c admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:28.243460 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5a08f-3da9-435b-82c9-7d95c09c098c admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:28.244196 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e5a08f-3da9-435b-82c9-7d95c09c098c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:45:28.244502 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e5a08f-3da9-435b-82c9-7d95c09c098c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:28.246119 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e5a08f-3da9-435b-82c9-7d95c09c098c admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:28.259959 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1167/2338] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:45:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:45:28.264080 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49175ee-cdea-411d-8d51-7fd3fb0548b4 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:28.275091 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49175ee-cdea-411d-8d51-7fd3fb0548b4 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:28.280666 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee642a-40b0-4748-91e0-7b1441d5115a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:28.281650 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee642a-40b0-4748-91e0-7b1441d5115a admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:28.281964 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee642a-40b0-4748-91e0-7b1441d5115a admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:28.282246 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee642a-40b0-4748-91e0-7b1441d5115a admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:28.283056 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee642a-40b0-4748-91e0-7b1441d5115a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:28.285491 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee642a-40b0-4748-91e0-7b1441d5115a admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:28.318234 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1172/2339] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:45:28 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:45:28.322320 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e5a08f-3da9-435b-82c9-7d95c09c098c admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:28.333236 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e5a08f-3da9-435b-82c9-7d95c09c098c admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:28.342277 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:28.427487 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9736712c-5344-4ba3-a878-d58491e427cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:28.428192 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736712c-5344-4ba3-a878-d58491e427cb admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:28.428483 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736712c-5344-4ba3-a878-d58491e427cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:28.428811 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736712c-5344-4ba3-a878-d58491e427cb admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:28.429499 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9736712c-5344-4ba3-a878-d58491e427cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:28.431652 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9736712c-5344-4ba3-a878-d58491e427cb admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:28.460107 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1168/2340] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:45:28 2022] POST /identity/v3/users => generated 287 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:45:28.464167 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee642a-40b0-4748-91e0-7b1441d5115a admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:28.475897 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee642a-40b0-4748-91e0-7b1441d5115a admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:28.481034 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb558a5-e009-4813-8993-d6e0df0a31d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:28.481708 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb558a5-e009-4813-8993-d6e0df0a31d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:28.481990 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb558a5-e009-4813-8993-d6e0df0a31d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:28.482268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb558a5-e009-4813-8993-d6e0df0a31d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:28.483029 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb558a5-e009-4813-8993-d6e0df0a31d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:28.484682 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb558a5-e009-4813-8993-d6e0df0a31d6 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:28.496886 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1173/2341] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:45:28.500701 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9736712c-5344-4ba3-a878-d58491e427cb admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:28.512552 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9736712c-5344-4ba3-a878-d58491e427cb admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:28.518228 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9efa4e-db2e-4411-9b75-16fccdbab863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:28.518942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9efa4e-db2e-4411-9b75-16fccdbab863 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:28.519247 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9efa4e-db2e-4411-9b75-16fccdbab863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:28.519497 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9efa4e-db2e-4411-9b75-16fccdbab863 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:28.520806 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9efa4e-db2e-4411-9b75-16fccdbab863 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:28.521962 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9efa4e-db2e-4411-9b75-16fccdbab863 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:28.533938 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1169/2342] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:45:28.538291 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb558a5-e009-4813-8993-d6e0df0a31d6 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:28.548912 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb558a5-e009-4813-8993-d6e0df0a31d6 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:28.554600 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29cb6ba-2a01-4ac8-8c6d-af9de9404fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:28.555702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cb6ba-2a01-4ac8-8c6d-af9de9404fa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:28.556164 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cb6ba-2a01-4ac8-8c6d-af9de9404fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:28.556448 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cb6ba-2a01-4ac8-8c6d-af9de9404fa1 admin admin] PATH_INFO: `/v3/projects/a4ff8845efbf4f06b405bb408c6a433d/users/6c1993819b8b4272a9bdcd7ca386139f/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:28.573868 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29cb6ba-2a01-4ac8-8c6d-af9de9404fa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4ff8845efbf4f06b405bb408c6a433d, user_id=6c1993819b8b4272a9bdcd7ca386139f, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:28.576702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29cb6ba-2a01-4ac8-8c6d-af9de9404fa1 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:28.591492 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1174/2343] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:45:28 2022] PUT /identity/v3/projects/a4ff8845efbf4f06b405bb408c6a433d/users/6c1993819b8b4272a9bdcd7ca386139f/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:28.598283 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da98c6-96e0-4682-ae42-778f4cc18bfe None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:28.599478 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da98c6-96e0-4682-ae42-778f4cc18bfe None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:28.600075 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da98c6-96e0-4682-ae42-778f4cc18bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:28.620497 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-38da98c6-96e0-4682-ae42-778f4cc18bfe None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:28.621780 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-38da98c6-96e0-4682-ae42-778f4cc18bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:28.686469 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1170/2344] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:28.699645 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29cb6ba-2a01-4ac8-8c6d-af9de9404fa1 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:28.701325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c29cb6ba-2a01-4ac8-8c6d-af9de9404fa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:28.766133 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29cb6ba-2a01-4ac8-8c6d-af9de9404fa1 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:28.769861 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03a0f5-ce0f-42dd-b085-e845b24d780b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:28.770527 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03a0f5-ce0f-42dd-b085-e845b24d780b service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:28.770795 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03a0f5-ce0f-42dd-b085-e845b24d780b service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:28.771108 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03a0f5-ce0f-42dd-b085-e845b24d780b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:28.781086 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03a0f5-ce0f-42dd-b085-e845b24d780b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:28.782531 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03a0f5-ce0f-42dd-b085-e845b24d780b service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:28.795616 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1175/2345] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:45:29.485644 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50a458-89ba-48d8-848e-4560fc2c5cc8 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:29.485939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50a458-89ba-48d8-848e-4560fc2c5cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:29.486228 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50a458-89ba-48d8-848e-4560fc2c5cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:29.534338 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-6a50a458-89ba-48d8-848e-4560fc2c5cc8 None None] MFA Rules not processed for user `6c1993819b8b4272a9bdcd7ca386139f`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:29.535784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-6a50a458-89ba-48d8-848e-4560fc2c5cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:29.657642 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1171/2346] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:29.662830 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03a0f5-ce0f-42dd-b085-e845b24d780b service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:29.673628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03a0f5-ce0f-42dd-b085-e845b24d780b service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:29.679943 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04268ea8-0443-469b-b844-9ee496c23fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:29.680865 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04268ea8-0443-469b-b844-9ee496c23fc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:29.681337 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04268ea8-0443-469b-b844-9ee496c23fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:29.681623 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04268ea8-0443-469b-b844-9ee496c23fc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:29.682343 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04268ea8-0443-469b-b844-9ee496c23fc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:29.684371 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04268ea8-0443-469b-b844-9ee496c23fc7 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:29.707578 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:29.736574 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1176/2347] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:45:29 2022] POST /identity/v3/projects => generated 373 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:45:29.741122 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a50a458-89ba-48d8-848e-4560fc2c5cc8 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:29.762320 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a50a458-89ba-48d8-848e-4560fc2c5cc8 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:29.787809 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:29.839048 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206dfdfc-0c34-493c-967a-65cc5ed5c656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:29.839779 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206dfdfc-0c34-493c-967a-65cc5ed5c656 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:29.840062 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206dfdfc-0c34-493c-967a-65cc5ed5c656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:29.840337 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206dfdfc-0c34-493c-967a-65cc5ed5c656 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:29.841022 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206dfdfc-0c34-493c-967a-65cc5ed5c656 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:29.842942 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206dfdfc-0c34-493c-967a-65cc5ed5c656 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:29.865991 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1172/2348] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:45:29 2022] POST /identity/v3/users => generated 293 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:45:29.870096 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04268ea8-0443-469b-b844-9ee496c23fc7 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:29.881522 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04268ea8-0443-469b-b844-9ee496c23fc7 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:29.886511 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b949f-7321-4084-b611-91afb43b11ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:29.887239 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b949f-7321-4084-b611-91afb43b11ff admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:29.887529 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b949f-7321-4084-b611-91afb43b11ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:29.887810 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b949f-7321-4084-b611-91afb43b11ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:29.888551 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b949f-7321-4084-b611-91afb43b11ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:29.890336 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b949f-7321-4084-b611-91afb43b11ff admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:29.903670 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1177/2349] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:29.908262 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206dfdfc-0c34-493c-967a-65cc5ed5c656 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:29.918877 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206dfdfc-0c34-493c-967a-65cc5ed5c656 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:29.924330 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf87d3b6-f485-42db-af29-ca25a031b6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:29.925240 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87d3b6-f485-42db-af29-ca25a031b6ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:29.925520 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87d3b6-f485-42db-af29-ca25a031b6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:29.925804 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87d3b6-f485-42db-af29-ca25a031b6ca admin admin] PATH_INFO: `/v3/projects/6ce3a62e1ba34467a358feaaf936692d/users/540d95f9b6434ebfa9487baf879ac8d7/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:29.943953 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf87d3b6-f485-42db-af29-ca25a031b6ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ce3a62e1ba34467a358feaaf936692d, user_id=540d95f9b6434ebfa9487baf879ac8d7, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:29.947053 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf87d3b6-f485-42db-af29-ca25a031b6ca admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:29.962236 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1173/2350] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:45:29 2022] PUT /identity/v3/projects/6ce3a62e1ba34467a358feaaf936692d/users/540d95f9b6434ebfa9487baf879ac8d7/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:29.966721 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b949f-7321-4084-b611-91afb43b11ff admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:29.985098 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b949f-7321-4084-b611-91afb43b11ff admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:30.057158 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bfd2de-3c54-4214-a06c-4f11743caeed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:30.059786 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfd2de-3c54-4214-a06c-4f11743caeed admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:30.060230 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfd2de-3c54-4214-a06c-4f11743caeed admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:30.060686 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfd2de-3c54-4214-a06c-4f11743caeed admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:30.061546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bfd2de-3c54-4214-a06c-4f11743caeed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:30.063751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bfd2de-3c54-4214-a06c-4f11743caeed admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:30.076159 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1178/2351] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:45:30.080357 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf87d3b6-f485-42db-af29-ca25a031b6ca admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:30.092200 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf87d3b6-f485-42db-af29-ca25a031b6ca admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:30.097995 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a4ae6-600d-465f-b3db-f91f6518f872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:30.104325 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a4ae6-600d-465f-b3db-f91f6518f872 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:30.104679 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a4ae6-600d-465f-b3db-f91f6518f872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:30.104979 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a4ae6-600d-465f-b3db-f91f6518f872 admin admin] PATH_INFO: `/v3/projects/6ce3a62e1ba34467a358feaaf936692d/users/540d95f9b6434ebfa9487baf879ac8d7/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:30.107384 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8a4ae6-600d-465f-b3db-f91f6518f872 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ce3a62e1ba34467a358feaaf936692d, user_id=540d95f9b6434ebfa9487baf879ac8d7, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:30.110202 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8a4ae6-600d-465f-b3db-f91f6518f872 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:30.121745 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1174/2352] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:45:30 2022] PUT /identity/v3/projects/6ce3a62e1ba34467a358feaaf936692d/users/540d95f9b6434ebfa9487baf879ac8d7/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:30.748791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54917b04-1e03-4df2-9ff2-7f7a8b6978db None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:30.749257 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54917b04-1e03-4df2-9ff2-7f7a8b6978db None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:30.749373 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54917b04-1e03-4df2-9ff2-7f7a8b6978db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:30.777595 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-54917b04-1e03-4df2-9ff2-7f7a8b6978db None None] MFA Rules not processed for user `540d95f9b6434ebfa9487baf879ac8d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:30.779139 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-54917b04-1e03-4df2-9ff2-7f7a8b6978db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:30.855067 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1179/2353] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:30 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:30.866657 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a4ae6-600d-465f-b3db-f91f6518f872 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:30.868450 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-5a8a4ae6-600d-465f-b3db-f91f6518f872 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:30.936098 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a4ae6-600d-465f-b3db-f91f6518f872 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:30.940552 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4e8bb-b616-4560-bded-1c48b3dbc519 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:30.941209 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4e8bb-b616-4560-bded-1c48b3dbc519 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:30.941495 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4e8bb-b616-4560-bded-1c48b3dbc519 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:30.941875 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4e8bb-b616-4560-bded-1c48b3dbc519 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:31.006639 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba4e8bb-b616-4560-bded-1c48b3dbc519 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:31.008255 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba4e8bb-b616-4560-bded-1c48b3dbc519 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:31.018370 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1175/2354] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:45:37.642377 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54917b04-1e03-4df2-9ff2-7f7a8b6978db None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:37.644969 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-54917b04-1e03-4df2-9ff2-7f7a8b6978db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:37.665309 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54917b04-1e03-4df2-9ff2-7f7a8b6978db None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:37.670724 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74755ad3-0506-4f8e-8d06-612503821908 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:37.671192 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74755ad3-0506-4f8e-8d06-612503821908 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:37.671695 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74755ad3-0506-4f8e-8d06-612503821908 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:37.672059 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74755ad3-0506-4f8e-8d06-612503821908 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:37.686064 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74755ad3-0506-4f8e-8d06-612503821908 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:37.687713 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74755ad3-0506-4f8e-8d06-612503821908 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:37.707573 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1180/2355] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:37 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:45:37.986520 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c1a8f-0435-4e70-a43d-660ba113a220 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:37.987602 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c1a8f-0435-4e70-a43d-660ba113a220 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:37.988190 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c1a8f-0435-4e70-a43d-660ba113a220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:38.013502 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-7d5c1a8f-0435-4e70-a43d-660ba113a220 None None] MFA Rules not processed for user `c9a3a2e49fee4ea08ca790006d0b29e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:38.015183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-7d5c1a8f-0435-4e70-a43d-660ba113a220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:38.044052 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:38.088078 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1176/2356] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:41.518648 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74755ad3-0506-4f8e-8d06-612503821908 service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:41.520847 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-74755ad3-0506-4f8e-8d06-612503821908 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:41.595946 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:41.614525 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74755ad3-0506-4f8e-8d06-612503821908 service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:41.621006 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01b0d68-544c-4750-a9fa-1f0ca78fc2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:41.622004 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b0d68-544c-4750-a9fa-1f0ca78fc2bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:41.622288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b0d68-544c-4750-a9fa-1f0ca78fc2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:41.622591 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b0d68-544c-4750-a9fa-1f0ca78fc2bf admin admin] PATH_INFO: `/v3/users/ed7321b77d954ceaaf14ca97c82be6a0` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:41.623895 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01b0d68-544c-4750-a9fa-1f0ca78fc2bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed7321b77d954ceaaf14ca97c82be6a0)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:41.626047 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01b0d68-544c-4750-a9fa-1f0ca78fc2bf admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:41.712585 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a01b0d68-544c-4750-a9fa-1f0ca78fc2bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed7321b77d954ceaaf14ca97c82be6a0'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:41.713945 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a01b0d68-544c-4750-a9fa-1f0ca78fc2bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed7321b77d954ceaaf14ca97c82be6a0'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:41.732872 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a01b0d68-544c-4750-a9fa-1f0ca78fc2bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed7321b77d954ceaaf14ca97c82be6a0'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:41.733641 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a01b0d68-544c-4750-a9fa-1f0ca78fc2bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed7321b77d954ceaaf14ca97c82be6a0'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:41.746532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a01b0d68-544c-4750-a9fa-1f0ca78fc2bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed7321b77d954ceaaf14ca97c82be6a0'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:41.761209 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1181/2357] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:45:41 2022] DELETE /identity/v3/users/ed7321b77d954ceaaf14ca97c82be6a0 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:41.765811 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb275017-829a-4dd4-863b-0701d48f18ef None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:41.766370 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb275017-829a-4dd4-863b-0701d48f18ef None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:41.766370 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb275017-829a-4dd4-863b-0701d48f18ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:41.794734 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-eb275017-829a-4dd4-863b-0701d48f18ef None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:41.796014 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-eb275017-829a-4dd4-863b-0701d48f18ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:41.893877 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1177/2358] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:41.907865 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01b0d68-544c-4750-a9fa-1f0ca78fc2bf admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:41.909421 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a01b0d68-544c-4750-a9fa-1f0ca78fc2bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:41.990277 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01b0d68-544c-4750-a9fa-1f0ca78fc2bf admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:42.008071 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8397c5-9cbb-49d2-8e36-cebb624655b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:42.009398 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8397c5-9cbb-49d2-8e36-cebb624655b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:42.009944 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8397c5-9cbb-49d2-8e36-cebb624655b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:42.010428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8397c5-9cbb-49d2-8e36-cebb624655b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:42.032234 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8397c5-9cbb-49d2-8e36-cebb624655b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:42.034100 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8397c5-9cbb-49d2-8e36-cebb624655b2 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:42.050409 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1182/2359] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:45:42.455681 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb275017-829a-4dd4-863b-0701d48f18ef None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:42.457482 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-eb275017-829a-4dd4-863b-0701d48f18ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:42.475546 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb275017-829a-4dd4-863b-0701d48f18ef None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:42.491140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e449b9f-3e5c-41d5-918f-45cc6007134b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:42.492043 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e449b9f-3e5c-41d5-918f-45cc6007134b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:42.492337 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e449b9f-3e5c-41d5-918f-45cc6007134b admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:42.492649 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e449b9f-3e5c-41d5-918f-45cc6007134b admin admin] PATH_INFO: `/v3/projects/ca51280a5d7547b8a5b3016461c1d432` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:42.493994 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e449b9f-3e5c-41d5-918f-45cc6007134b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca51280a5d7547b8a5b3016461c1d432)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:42.496646 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e449b9f-3e5c-41d5-918f-45cc6007134b admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:42.570019 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-9e449b9f-3e5c-41d5-918f-45cc6007134b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca51280a5d7547b8a5b3016461c1d432', 'request_id': 'req-9e449b9f-3e5c-41d5-918f-45cc6007134b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:42.580996 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-9e449b9f-3e5c-41d5-918f-45cc6007134b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca51280a5d7547b8a5b3016461c1d432', 'request_id': 'req-9e449b9f-3e5c-41d5-918f-45cc6007134b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:42.588534 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-9e449b9f-3e5c-41d5-918f-45cc6007134b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca51280a5d7547b8a5b3016461c1d432', 'request_id': 'req-9e449b9f-3e5c-41d5-918f-45cc6007134b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:42.606435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-9e449b9f-3e5c-41d5-918f-45cc6007134b admin admin] The token cache is being invalidate because project ca51280a5d7547b8a5b3016461c1d432 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:45:42.606638 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-9e449b9f-3e5c-41d5-918f-45cc6007134b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:42.607361 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-9e449b9f-3e5c-41d5-918f-45cc6007134b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:42.608975 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1178/2360] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:45:42 2022] DELETE /identity/v3/projects/ca51280a5d7547b8a5b3016461c1d432 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:42.613998 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece44337-7876-453c-bee5-29f07ad120a7 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:42.615198 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece44337-7876-453c-bee5-29f07ad120a7 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:42.615198 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece44337-7876-453c-bee5-29f07ad120a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:42.635115 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-ece44337-7876-453c-bee5-29f07ad120a7 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:42.636355 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ece44337-7876-453c-bee5-29f07ad120a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:42.709100 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1183/2361] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:45:42.718499 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b39eb-6369-42f3-bf30-e3650076c05a None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:42.718810 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b39eb-6369-42f3-bf30-e3650076c05a None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:42.719138 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b39eb-6369-42f3-bf30-e3650076c05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:42.739214 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-f55b39eb-6369-42f3-bf30-e3650076c05a None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:42.741077 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f55b39eb-6369-42f3-bf30-e3650076c05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:42.749677 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1179/2362] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:42.758108 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece44337-7876-453c-bee5-29f07ad120a7 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:42.768374 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece44337-7876-453c-bee5-29f07ad120a7 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:42.773457 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827807f4-543a-4e36-8dfa-ec7dafd3526e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:42.774139 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827807f4-543a-4e36-8dfa-ec7dafd3526e admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:42.774440 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827807f4-543a-4e36-8dfa-ec7dafd3526e admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:42.774708 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827807f4-543a-4e36-8dfa-ec7dafd3526e admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:42.775533 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827807f4-543a-4e36-8dfa-ec7dafd3526e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:45:42.775849 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827807f4-543a-4e36-8dfa-ec7dafd3526e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:42.777603 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827807f4-543a-4e36-8dfa-ec7dafd3526e admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:42.792538 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1184/2363] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:45:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:45:42.798506 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55b39eb-6369-42f3-bf30-e3650076c05a None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:42.809222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55b39eb-6369-42f3-bf30-e3650076c05a None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:42.814444 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cbdb6b-d0af-4bc8-bb86-2fbf13a9d88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:42.815183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbdb6b-d0af-4bc8-bb86-2fbf13a9d88a admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:42.815465 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbdb6b-d0af-4bc8-bb86-2fbf13a9d88a admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:42.815745 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbdb6b-d0af-4bc8-bb86-2fbf13a9d88a admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:42.816446 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cbdb6b-d0af-4bc8-bb86-2fbf13a9d88a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:42.819065 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cbdb6b-d0af-4bc8-bb86-2fbf13a9d88a admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:42.855582 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1180/2364] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:45:42 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:45:42.860084 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827807f4-543a-4e36-8dfa-ec7dafd3526e admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:42.865237 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:42.872502 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827807f4-543a-4e36-8dfa-ec7dafd3526e admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:42.933951 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b08dbca-b66e-4542-b366-b8bc3ab5d8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:42.934624 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08dbca-b66e-4542-b366-b8bc3ab5d8e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:42.934912 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08dbca-b66e-4542-b366-b8bc3ab5d8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:42.935217 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08dbca-b66e-4542-b366-b8bc3ab5d8e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:42.935920 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b08dbca-b66e-4542-b366-b8bc3ab5d8e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:42.938057 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b08dbca-b66e-4542-b366-b8bc3ab5d8e1 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:42.963982 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1185/2365] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:45:42 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:45:42.968147 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cbdb6b-d0af-4bc8-bb86-2fbf13a9d88a admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:42.983469 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cbdb6b-d0af-4bc8-bb86-2fbf13a9d88a admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:42.988517 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27458a61-0e11-42d2-a02b-5c88d247b81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:42.989190 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27458a61-0e11-42d2-a02b-5c88d247b81b admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:42.989476 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27458a61-0e11-42d2-a02b-5c88d247b81b admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:42.989904 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27458a61-0e11-42d2-a02b-5c88d247b81b admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:42.991615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27458a61-0e11-42d2-a02b-5c88d247b81b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:42.993332 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27458a61-0e11-42d2-a02b-5c88d247b81b admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:43.005858 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1181/2366] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:43.010219 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b08dbca-b66e-4542-b366-b8bc3ab5d8e1 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:43.024776 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b08dbca-b66e-4542-b366-b8bc3ab5d8e1 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:43.033440 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf124d-3f68-4b1b-90b8-3427040a4658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:43.034126 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf124d-3f68-4b1b-90b8-3427040a4658 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:43.034413 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf124d-3f68-4b1b-90b8-3427040a4658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:43.034692 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf124d-3f68-4b1b-90b8-3427040a4658 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:43.035476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf124d-3f68-4b1b-90b8-3427040a4658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:43.037150 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf124d-3f68-4b1b-90b8-3427040a4658 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:43.047935 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1186/2367] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:43.052089 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27458a61-0e11-42d2-a02b-5c88d247b81b admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:43.062884 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27458a61-0e11-42d2-a02b-5c88d247b81b admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:43.067616 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f4d089-10bb-4118-99b5-7da87cf80b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:43.069020 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f4d089-10bb-4118-99b5-7da87cf80b1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:43.069410 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f4d089-10bb-4118-99b5-7da87cf80b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:43.069693 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f4d089-10bb-4118-99b5-7da87cf80b1a admin admin] PATH_INFO: `/v3/projects/58cd4da4b6a0405dac6d58b9b10a99c0/users/c2085ea4ef85442ba548c952a910b787/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:43.084815 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:43.088528 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f4d089-10bb-4118-99b5-7da87cf80b1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=58cd4da4b6a0405dac6d58b9b10a99c0, user_id=c2085ea4ef85442ba548c952a910b787, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:43.091369 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f4d089-10bb-4118-99b5-7da87cf80b1a admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:43.109838 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1182/2368] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:45:43 2022] PUT /identity/v3/projects/58cd4da4b6a0405dac6d58b9b10a99c0/users/c2085ea4ef85442ba548c952a910b787/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:43.120844 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3abfb7-fad8-4a45-a3e3-e78482d68535 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:43.121082 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3abfb7-fad8-4a45-a3e3-e78482d68535 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:43.121383 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3abfb7-fad8-4a45-a3e3-e78482d68535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:43.155154 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-3d3abfb7-fad8-4a45-a3e3-e78482d68535 None None] MFA Rules not processed for user `c2085ea4ef85442ba548c952a910b787`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:43.156395 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-3d3abfb7-fad8-4a45-a3e3-e78482d68535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:43.224232 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1187/2369] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:43.235892 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f4d089-10bb-4118-99b5-7da87cf80b1a admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:43.237820 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-a2f4d089-10bb-4118-99b5-7da87cf80b1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:43.298415 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f4d089-10bb-4118-99b5-7da87cf80b1a admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:43.303549 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d2bc48-df9a-4f4f-a84a-761c660e7c80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:43.304374 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2bc48-df9a-4f4f-a84a-761c660e7c80 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:43.304880 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2bc48-df9a-4f4f-a84a-761c660e7c80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:43.305027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2bc48-df9a-4f4f-a84a-761c660e7c80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:43.315094 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d2bc48-df9a-4f4f-a84a-761c660e7c80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:43.316532 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d2bc48-df9a-4f4f-a84a-761c660e7c80 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:43.326379 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1183/2370] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:45:53.221416 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=65,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:53.322062 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=29,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:54.600764 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3abfb7-fad8-4a45-a3e3-e78482d68535 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:54.602336 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-3d3abfb7-fad8-4a45-a3e3-e78482d68535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:54.676555 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3abfb7-fad8-4a45-a3e3-e78482d68535 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:54.681516 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30585e1-a1f9-41af-b75f-e246541e16a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:54.682421 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30585e1-a1f9-41af-b75f-e246541e16a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:54.682703 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30585e1-a1f9-41af-b75f-e246541e16a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:54.682979 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30585e1-a1f9-41af-b75f-e246541e16a7 admin admin] PATH_INFO: `/v3/users/6c1993819b8b4272a9bdcd7ca386139f` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:54.684301 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30585e1-a1f9-41af-b75f-e246541e16a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c1993819b8b4272a9bdcd7ca386139f)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:54.686220 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30585e1-a1f9-41af-b75f-e246541e16a7 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:54.741041 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d30585e1-a1f9-41af-b75f-e246541e16a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c1993819b8b4272a9bdcd7ca386139f'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:54.741720 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d30585e1-a1f9-41af-b75f-e246541e16a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c1993819b8b4272a9bdcd7ca386139f'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:54.754310 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d30585e1-a1f9-41af-b75f-e246541e16a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c1993819b8b4272a9bdcd7ca386139f'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:54.754969 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d30585e1-a1f9-41af-b75f-e246541e16a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c1993819b8b4272a9bdcd7ca386139f'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:54.763464 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d30585e1-a1f9-41af-b75f-e246541e16a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c1993819b8b4272a9bdcd7ca386139f'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:54.775988 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1188/2371] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:45:54 2022] DELETE /identity/v3/users/6c1993819b8b4272a9bdcd7ca386139f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:54.951299 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d2bc48-df9a-4f4f-a84a-761c660e7c80 service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:54.963702 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d2bc48-df9a-4f4f-a84a-761c660e7c80 service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:55.025685 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618cd5aa-0125-45b5-ba19-b5d372d69903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:55.026431 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618cd5aa-0125-45b5-ba19-b5d372d69903 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:55.026713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618cd5aa-0125-45b5-ba19-b5d372d69903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:55.027008 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618cd5aa-0125-45b5-ba19-b5d372d69903 admin admin] PATH_INFO: `/v3/projects/a4ff8845efbf4f06b405bb408c6a433d` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:55.028202 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618cd5aa-0125-45b5-ba19-b5d372d69903 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4ff8845efbf4f06b405bb408c6a433d)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:55.031193 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618cd5aa-0125-45b5-ba19-b5d372d69903 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:55.081602 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-618cd5aa-0125-45b5-ba19-b5d372d69903 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4ff8845efbf4f06b405bb408c6a433d', 'request_id': 'req-618cd5aa-0125-45b5-ba19-b5d372d69903'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:55.089075 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-618cd5aa-0125-45b5-ba19-b5d372d69903 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4ff8845efbf4f06b405bb408c6a433d', 'request_id': 'req-618cd5aa-0125-45b5-ba19-b5d372d69903'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:55.095568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-618cd5aa-0125-45b5-ba19-b5d372d69903 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4ff8845efbf4f06b405bb408c6a433d', 'request_id': 'req-618cd5aa-0125-45b5-ba19-b5d372d69903'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:55.106686 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-618cd5aa-0125-45b5-ba19-b5d372d69903 admin admin] The token cache is being invalidate because project a4ff8845efbf4f06b405bb408c6a433d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:45:55.106976 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-618cd5aa-0125-45b5-ba19-b5d372d69903 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:55.107636 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-618cd5aa-0125-45b5-ba19-b5d372d69903 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:55.109173 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1184/2372] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:45:54 2022] DELETE /identity/v3/projects/a4ff8845efbf4f06b405bb408c6a433d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:55.113350 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30585e1-a1f9-41af-b75f-e246541e16a7 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:55.114733 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d30585e1-a1f9-41af-b75f-e246541e16a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:55.128199 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30585e1-a1f9-41af-b75f-e246541e16a7 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:55.187548 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633b8f3-8853-4084-8c0f-330e1b24d385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:55.188433 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633b8f3-8853-4084-8c0f-330e1b24d385 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:55.188715 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633b8f3-8853-4084-8c0f-330e1b24d385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:55.188991 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633b8f3-8853-4084-8c0f-330e1b24d385 admin admin] PATH_INFO: `/v3/users/540d95f9b6434ebfa9487baf879ac8d7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:55.190198 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3633b8f3-8853-4084-8c0f-330e1b24d385 admin admin] RBAC: Authorizing `identity:delete_user(user_id=540d95f9b6434ebfa9487baf879ac8d7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:55.192193 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3633b8f3-8853-4084-8c0f-330e1b24d385 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:55.214742 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:55.241612 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3633b8f3-8853-4084-8c0f-330e1b24d385 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '540d95f9b6434ebfa9487baf879ac8d7'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:55.242296 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3633b8f3-8853-4084-8c0f-330e1b24d385 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '540d95f9b6434ebfa9487baf879ac8d7'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:55.259872 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3633b8f3-8853-4084-8c0f-330e1b24d385 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '540d95f9b6434ebfa9487baf879ac8d7'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:55.260930 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3633b8f3-8853-4084-8c0f-330e1b24d385 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '540d95f9b6434ebfa9487baf879ac8d7'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:55.269063 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3633b8f3-8853-4084-8c0f-330e1b24d385 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '540d95f9b6434ebfa9487baf879ac8d7'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:55.282376 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1189/2373] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:45:55 2022] DELETE /identity/v3/users/540d95f9b6434ebfa9487baf879ac8d7 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:55.455673 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618cd5aa-0125-45b5-ba19-b5d372d69903 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:55.457325 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-618cd5aa-0125-45b5-ba19-b5d372d69903 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:55.472516 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618cd5aa-0125-45b5-ba19-b5d372d69903 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:55.530228 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ad340-7c66-4a11-baa8-0f4ababf0fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:55.531127 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ad340-7c66-4a11-baa8-0f4ababf0fd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:55.531413 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ad340-7c66-4a11-baa8-0f4ababf0fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:55.531689 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ad340-7c66-4a11-baa8-0f4ababf0fd0 admin admin] PATH_INFO: `/v3/projects/6ce3a62e1ba34467a358feaaf936692d` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:55.532920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696ad340-7c66-4a11-baa8-0f4ababf0fd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ce3a62e1ba34467a358feaaf936692d)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:55.534767 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696ad340-7c66-4a11-baa8-0f4ababf0fd0 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:55.570099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:55.584276 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-696ad340-7c66-4a11-baa8-0f4ababf0fd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ce3a62e1ba34467a358feaaf936692d', 'request_id': 'req-696ad340-7c66-4a11-baa8-0f4ababf0fd0'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:55.593517 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-696ad340-7c66-4a11-baa8-0f4ababf0fd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ce3a62e1ba34467a358feaaf936692d', 'request_id': 'req-696ad340-7c66-4a11-baa8-0f4ababf0fd0'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:55.602007 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-696ad340-7c66-4a11-baa8-0f4ababf0fd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ce3a62e1ba34467a358feaaf936692d', 'request_id': 'req-696ad340-7c66-4a11-baa8-0f4ababf0fd0'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:55.613270 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-696ad340-7c66-4a11-baa8-0f4ababf0fd0 admin admin] The token cache is being invalidate because project 6ce3a62e1ba34467a358feaaf936692d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:45:55.613564 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-696ad340-7c66-4a11-baa8-0f4ababf0fd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:55.614227 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-696ad340-7c66-4a11-baa8-0f4ababf0fd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:55.615810 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1185/2374] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:45:55 2022] DELETE /identity/v3/projects/6ce3a62e1ba34467a358feaaf936692d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:55.620805 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1e312-eb2f-4feb-9fd8-b5c57ed0eef4 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:55.621329 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1e312-eb2f-4feb-9fd8-b5c57ed0eef4 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:55.621453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1e312-eb2f-4feb-9fd8-b5c57ed0eef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:55.641198 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-d4a1e312-eb2f-4feb-9fd8-b5c57ed0eef4 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:55.642414 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d4a1e312-eb2f-4feb-9fd8-b5c57ed0eef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:55.709854 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1190/2375] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:55.712561 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c0429-95dc-4a6c-8f7c-33d36823d5b4 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:55.712850 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c0429-95dc-4a6c-8f7c-33d36823d5b4 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:55.713128 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c0429-95dc-4a6c-8f7c-33d36823d5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:55.733442 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-857c0429-95dc-4a6c-8f7c-33d36823d5b4 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:55.734650 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-857c0429-95dc-4a6c-8f7c-33d36823d5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:55.743974 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1186/2376] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:55.745043 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a1e312-eb2f-4feb-9fd8-b5c57ed0eef4 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:55.746543 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d4a1e312-eb2f-4feb-9fd8-b5c57ed0eef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:55.748065 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c0429-95dc-4a6c-8f7c-33d36823d5b4 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:55.759121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a1e312-eb2f-4feb-9fd8-b5c57ed0eef4 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:55.763693 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a81c71-af34-437f-8df1-177d352e18e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:55.764563 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a81c71-af34-437f-8df1-177d352e18e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:55.764844 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a81c71-af34-437f-8df1-177d352e18e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:55.764955 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c0429-95dc-4a6c-8f7c-33d36823d5b4 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:55.765117 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a81c71-af34-437f-8df1-177d352e18e4 admin admin] PATH_INFO: `/v3/users/c2085ea4ef85442ba548c952a910b787` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:55.766309 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a81c71-af34-437f-8df1-177d352e18e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2085ea4ef85442ba548c952a910b787)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:55.768274 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a81c71-af34-437f-8df1-177d352e18e4 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:55.769727 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e78633-e4cf-4ce7-a4a8-351ee0bd66a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:55.770386 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e78633-e4cf-4ce7-a4a8-351ee0bd66a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:55.770666 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e78633-e4cf-4ce7-a4a8-351ee0bd66a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:55.770949 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e78633-e4cf-4ce7-a4a8-351ee0bd66a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:55.771713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e78633-e4cf-4ce7-a4a8-351ee0bd66a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:45:55.772022 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e78633-e4cf-4ce7-a4a8-351ee0bd66a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:55.773609 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e78633-e4cf-4ce7-a4a8-351ee0bd66a0 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:55.787172 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1187/2377] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:45:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:45:55.791308 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e78633-e4cf-4ce7-a4a8-351ee0bd66a0 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:55.803359 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e78633-e4cf-4ce7-a4a8-351ee0bd66a0 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:55.809242 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb451e-1cdc-4f07-91f1-5857d4936e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:55.810083 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb451e-1cdc-4f07-91f1-5857d4936e77 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:55.810364 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb451e-1cdc-4f07-91f1-5857d4936e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:55.810691 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb451e-1cdc-4f07-91f1-5857d4936e77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:55.811453 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cb451e-1cdc-4f07-91f1-5857d4936e77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:55.813399 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cb451e-1cdc-4f07-91f1-5857d4936e77 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:55.822001 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d8a81c71-af34-437f-8df1-177d352e18e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2085ea4ef85442ba548c952a910b787'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:55.822852 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d8a81c71-af34-437f-8df1-177d352e18e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2085ea4ef85442ba548c952a910b787'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:55.839906 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d8a81c71-af34-437f-8df1-177d352e18e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2085ea4ef85442ba548c952a910b787'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:55.840767 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d8a81c71-af34-437f-8df1-177d352e18e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2085ea4ef85442ba548c952a910b787'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:55.846024 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1188/2378] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:45:55 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:45:55.848843 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d8a81c71-af34-437f-8df1-177d352e18e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2085ea4ef85442ba548c952a910b787'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:55.850154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb451e-1cdc-4f07-91f1-5857d4936e77 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:55.851896 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-a1cb451e-1cdc-4f07-91f1-5857d4936e77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:55.862158 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1191/2379] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:45:55 2022] DELETE /identity/v3/users/c2085ea4ef85442ba548c952a910b787 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:55.866689 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a284a-71fc-4ebe-b97e-0b5f37614809 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:55.866981 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a284a-71fc-4ebe-b97e-0b5f37614809 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:55.867293 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a284a-71fc-4ebe-b97e-0b5f37614809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:55.886309 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-9b3a284a-71fc-4ebe-b97e-0b5f37614809 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:55.887574 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-9b3a284a-71fc-4ebe-b97e-0b5f37614809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:55.893238 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:55.932974 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb451e-1cdc-4f07-91f1-5857d4936e77 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:55.963447 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1192/2380] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:45:55.975195 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3a284a-71fc-4ebe-b97e-0b5f37614809 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:55.976574 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-9b3a284a-71fc-4ebe-b97e-0b5f37614809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:56.002335 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc5ad29-5beb-42ba-9320-13b756c558fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:56.003278 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5ad29-5beb-42ba-9320-13b756c558fb admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:56.003637 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5ad29-5beb-42ba-9320-13b756c558fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:56.003996 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5ad29-5beb-42ba-9320-13b756c558fb admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:56.004810 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc5ad29-5beb-42ba-9320-13b756c558fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:56.006762 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc5ad29-5beb-42ba-9320-13b756c558fb admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:56.029792 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1189/2381] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:45:55 2022] POST /identity/v3/users => generated 283 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:45:56.034044 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc5ad29-5beb-42ba-9320-13b756c558fb admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:56.038732 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3a284a-71fc-4ebe-b97e-0b5f37614809 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:56.042939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d94d21-058f-43d6-b27e-d873e650549e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:56.043380 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d94d21-058f-43d6-b27e-d873e650549e service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:56.043666 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d94d21-058f-43d6-b27e-d873e650549e service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:56.043943 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d94d21-058f-43d6-b27e-d873e650549e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:56.045701 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc5ad29-5beb-42ba-9320-13b756c558fb admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:56.050724 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288bdea-a89f-41e0-bece-a343cf25adf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:56.051425 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288bdea-a89f-41e0-bece-a343cf25adf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:56.051714 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288bdea-a89f-41e0-bece-a343cf25adf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:56.051992 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288bdea-a89f-41e0-bece-a343cf25adf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:56.052838 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b288bdea-a89f-41e0-bece-a343cf25adf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:56.054233 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d94d21-058f-43d6-b27e-d873e650549e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:56.054471 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b288bdea-a89f-41e0-bece-a343cf25adf9 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:56.056260 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d94d21-058f-43d6-b27e-d873e650549e service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:56.067056 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1190/2382] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:56.068172 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1193/2383] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:45:56.071538 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288bdea-a89f-41e0-bece-a343cf25adf9 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:56.079182 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:56.083799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288bdea-a89f-41e0-bece-a343cf25adf9 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:56.094631 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ba8ca-e9a7-4a42-a471-22f23155af5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:56.095370 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ba8ca-e9a7-4a42-a471-22f23155af5f admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:56.095811 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ba8ca-e9a7-4a42-a471-22f23155af5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:56.096106 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ba8ca-e9a7-4a42-a471-22f23155af5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:56.096857 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ba8ca-e9a7-4a42-a471-22f23155af5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:56.098576 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ba8ca-e9a7-4a42-a471-22f23155af5f admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:56.115086 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1191/2384] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:56.116645 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d94d21-058f-43d6-b27e-d873e650549e service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:56.132175 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d94d21-058f-43d6-b27e-d873e650549e service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:56.140522 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dcc49c-d867-49f1-9d4a-57a956c91df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:56.141552 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcc49c-d867-49f1-9d4a-57a956c91df4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:56.141940 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcc49c-d867-49f1-9d4a-57a956c91df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:56.142323 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcc49c-d867-49f1-9d4a-57a956c91df4 admin admin] PATH_INFO: `/v3/projects/f516bcee07b54b2b8dae1cf0ed180746/users/0c8244e8b8bd43489c87a25c4c8c6240/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:56.159819 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dcc49c-d867-49f1-9d4a-57a956c91df4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f516bcee07b54b2b8dae1cf0ed180746, user_id=0c8244e8b8bd43489c87a25c4c8c6240, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:56.162867 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dcc49c-d867-49f1-9d4a-57a956c91df4 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:56.183534 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1194/2385] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:45:56 2022] PUT /identity/v3/projects/f516bcee07b54b2b8dae1cf0ed180746/users/0c8244e8b8bd43489c87a25c4c8c6240/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:56.194499 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ead179-2975-4a59-b2b2-540e92f9c0c8 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:56.194792 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ead179-2975-4a59-b2b2-540e92f9c0c8 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:56.196657 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ead179-2975-4a59-b2b2-540e92f9c0c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:56.230243 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-12ead179-2975-4a59-b2b2-540e92f9c0c8 None None] MFA Rules not processed for user `0c8244e8b8bd43489c87a25c4c8c6240`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:56.232160 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-12ead179-2975-4a59-b2b2-540e92f9c0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:56.318858 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1192/2386] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:45:56 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:56.339709 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dcc49c-d867-49f1-9d4a-57a956c91df4 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:56.342015 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-25dcc49c-d867-49f1-9d4a-57a956c91df4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:56.376182 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ead179-2975-4a59-b2b2-540e92f9c0c8 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:56.378264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-12ead179-2975-4a59-b2b2-540e92f9c0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:56.420229 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dcc49c-d867-49f1-9d4a-57a956c91df4 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:56.423957 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfafc1d-9a07-4edf-aac1-cb6976299df7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:56.424677 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfafc1d-9a07-4edf-aac1-cb6976299df7 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:56.424965 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfafc1d-9a07-4edf-aac1-cb6976299df7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:56.425240 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfafc1d-9a07-4edf-aac1-cb6976299df7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:56.434596 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfafc1d-9a07-4edf-aac1-cb6976299df7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:56.436102 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfafc1d-9a07-4edf-aac1-cb6976299df7 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:56.445366 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1195/2387] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:56 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:45:56.460922 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ead179-2975-4a59-b2b2-540e92f9c0c8 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:56.469792 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14794b-2c53-4778-bf53-7b1332aba5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:56.470658 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14794b-2c53-4778-bf53-7b1332aba5e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:56.470937 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14794b-2c53-4778-bf53-7b1332aba5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:56.472474 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14794b-2c53-4778-bf53-7b1332aba5e5 admin admin] PATH_INFO: `/v3/projects/58cd4da4b6a0405dac6d58b9b10a99c0` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:56.472474 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14794b-2c53-4778-bf53-7b1332aba5e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58cd4da4b6a0405dac6d58b9b10a99c0)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:56.474354 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14794b-2c53-4778-bf53-7b1332aba5e5 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:56.525041 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-4e14794b-2c53-4778-bf53-7b1332aba5e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58cd4da4b6a0405dac6d58b9b10a99c0', 'request_id': 'req-4e14794b-2c53-4778-bf53-7b1332aba5e5'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:56.533027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-4e14794b-2c53-4778-bf53-7b1332aba5e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58cd4da4b6a0405dac6d58b9b10a99c0', 'request_id': 'req-4e14794b-2c53-4778-bf53-7b1332aba5e5'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:56.540515 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-4e14794b-2c53-4778-bf53-7b1332aba5e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58cd4da4b6a0405dac6d58b9b10a99c0', 'request_id': 'req-4e14794b-2c53-4778-bf53-7b1332aba5e5'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:56.552615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-4e14794b-2c53-4778-bf53-7b1332aba5e5 admin admin] The token cache is being invalidate because project 58cd4da4b6a0405dac6d58b9b10a99c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:45:56.552839 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-4e14794b-2c53-4778-bf53-7b1332aba5e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:56.553515 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-4e14794b-2c53-4778-bf53-7b1332aba5e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:56.555190 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1193/2388] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:45:56 2022] DELETE /identity/v3/projects/58cd4da4b6a0405dac6d58b9b10a99c0 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:46:01.549984 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfafc1d-9a07-4edf-aac1-cb6976299df7 service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:01.551468 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-abfafc1d-9a07-4edf-aac1-cb6976299df7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:01.603195 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:46:01.661462 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfafc1d-9a07-4edf-aac1-cb6976299df7 service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:01.727891 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7baea05-3f45-427b-838e-5384d48d6dff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4128111a9aa450abb2146a5d9926dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:01.728668 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7baea05-3f45-427b-838e-5384d48d6dff service placement] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:01.729005 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7baea05-3f45-427b-838e-5384d48d6dff service placement] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:01.729392 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7baea05-3f45-427b-838e-5384d48d6dff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:01.731733 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-b7baea05-3f45-427b-838e-5384d48d6dff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:01.744411 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7baea05-3f45-427b-838e-5384d48d6dff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:01.746033 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7baea05-3f45-427b-838e-5384d48d6dff service placement] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:01.756192 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1196/2389] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:46:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:46:06.549382 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:46:11.751683 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=52 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:46:21.183804 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14794b-2c53-4778-bf53-7b1332aba5e5 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:21.185265 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-4e14794b-2c53-4778-bf53-7b1332aba5e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:21.247691 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14794b-2c53-4778-bf53-7b1332aba5e5 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:21.251301 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede1a553-6ffc-4246-a543-7d40f03119c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:21.251982 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1a553-6ffc-4246-a543-7d40f03119c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:21.252266 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1a553-6ffc-4246-a543-7d40f03119c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:21.252544 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1a553-6ffc-4246-a543-7d40f03119c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:21.254352 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ede1a553-6ffc-4246-a543-7d40f03119c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:21.322011 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede1a553-6ffc-4246-a543-7d40f03119c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:21.323527 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede1a553-6ffc-4246-a543-7d40f03119c9 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:21.334009 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1194/2390] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:46:21 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:46:31.329257 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=50 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:46:48.301484 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7baea05-3f45-427b-838e-5384d48d6dff service placement] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:48.302873 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-b7baea05-3f45-427b-838e-5384d48d6dff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:48.369335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7baea05-3f45-427b-838e-5384d48d6dff service placement] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:48.375777 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03706d89-ccd9-4e8a-b347-2b0aa9def888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:48.376674 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03706d89-ccd9-4e8a-b347-2b0aa9def888 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:48.376961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03706d89-ccd9-4e8a-b347-2b0aa9def888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:48.377241 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03706d89-ccd9-4e8a-b347-2b0aa9def888 admin admin] PATH_INFO: `/v3/users/0c8244e8b8bd43489c87a25c4c8c6240` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:48.378485 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03706d89-ccd9-4e8a-b347-2b0aa9def888 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c8244e8b8bd43489c87a25c4c8c6240)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:48.380597 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03706d89-ccd9-4e8a-b347-2b0aa9def888 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:48.430521 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-03706d89-ccd9-4e8a-b347-2b0aa9def888 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c8244e8b8bd43489c87a25c4c8c6240'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:46:48.431200 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-03706d89-ccd9-4e8a-b347-2b0aa9def888 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c8244e8b8bd43489c87a25c4c8c6240'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:46:48.444034 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-03706d89-ccd9-4e8a-b347-2b0aa9def888 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c8244e8b8bd43489c87a25c4c8c6240'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:46:48.444716 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-03706d89-ccd9-4e8a-b347-2b0aa9def888 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c8244e8b8bd43489c87a25c4c8c6240'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:46:48.453405 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-03706d89-ccd9-4e8a-b347-2b0aa9def888 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c8244e8b8bd43489c87a25c4c8c6240'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:46:48.465536 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1197/2391] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:46:48 2022] DELETE /identity/v3/users/0c8244e8b8bd43489c87a25c4c8c6240 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:46:48.470131 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4570fc9-a3da-428f-adb8-861459fa895e None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:48.470602 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4570fc9-a3da-428f-adb8-861459fa895e None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:48.470765 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4570fc9-a3da-428f-adb8-861459fa895e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:48.487770 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-c4570fc9-a3da-428f-adb8-861459fa895e None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:46:48.488978 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c4570fc9-a3da-428f-adb8-861459fa895e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:48.547459 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1195/2392] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:46:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:46:48.557046 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03706d89-ccd9-4e8a-b347-2b0aa9def888 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:48.558458 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-03706d89-ccd9-4e8a-b347-2b0aa9def888 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:48.615710 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03706d89-ccd9-4e8a-b347-2b0aa9def888 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:48.619434 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c3afef-3e6e-4681-b40c-fdaaace9fb69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:48.620105 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3afef-3e6e-4681-b40c-fdaaace9fb69 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:48.620387 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3afef-3e6e-4681-b40c-fdaaace9fb69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:48.620659 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3afef-3e6e-4681-b40c-fdaaace9fb69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:48.630207 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c3afef-3e6e-4681-b40c-fdaaace9fb69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:48.631663 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c3afef-3e6e-4681-b40c-fdaaace9fb69 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:48.641514 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1198/2393] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:46:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:46:48.970829 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4570fc9-a3da-428f-adb8-861459fa895e None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:48.973234 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c4570fc9-a3da-428f-adb8-861459fa895e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:48.989527 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4570fc9-a3da-428f-adb8-861459fa895e None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:48.995510 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1bcf65-ea48-4233-b091-f7a12b822408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:48.996417 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bcf65-ea48-4233-b091-f7a12b822408 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:48.996701 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bcf65-ea48-4233-b091-f7a12b822408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:48.996977 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bcf65-ea48-4233-b091-f7a12b822408 admin admin] PATH_INFO: `/v3/projects/f516bcee07b54b2b8dae1cf0ed180746` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:48.998237 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1bcf65-ea48-4233-b091-f7a12b822408 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f516bcee07b54b2b8dae1cf0ed180746)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:49.000221 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1bcf65-ea48-4233-b091-f7a12b822408 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:49.059826 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ca1bcf65-ea48-4233-b091-f7a12b822408 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f516bcee07b54b2b8dae1cf0ed180746', 'request_id': 'req-ca1bcf65-ea48-4233-b091-f7a12b822408'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:46:49.068172 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ca1bcf65-ea48-4233-b091-f7a12b822408 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f516bcee07b54b2b8dae1cf0ed180746', 'request_id': 'req-ca1bcf65-ea48-4233-b091-f7a12b822408'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:46:49.075555 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ca1bcf65-ea48-4233-b091-f7a12b822408 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f516bcee07b54b2b8dae1cf0ed180746', 'request_id': 'req-ca1bcf65-ea48-4233-b091-f7a12b822408'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:46:49.086928 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ca1bcf65-ea48-4233-b091-f7a12b822408 admin admin] The token cache is being invalidate because project f516bcee07b54b2b8dae1cf0ed180746 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:46:49.087240 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ca1bcf65-ea48-4233-b091-f7a12b822408 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:46:49.087987 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ca1bcf65-ea48-4233-b091-f7a12b822408 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:46:49.089570 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1196/2394] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:46:48 2022] DELETE /identity/v3/projects/f516bcee07b54b2b8dae1cf0ed180746 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:46:49.095040 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674fd6a3-3261-45f8-ab98-250f55d282c5 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:49.095335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674fd6a3-3261-45f8-ab98-250f55d282c5 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:49.095613 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674fd6a3-3261-45f8-ab98-250f55d282c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:49.114041 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-674fd6a3-3261-45f8-ab98-250f55d282c5 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:46:49.115300 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-674fd6a3-3261-45f8-ab98-250f55d282c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:49.132516 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:46:49.179555 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1199/2395] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:46:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:46:49.187514 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152cb8ab-daef-43bf-a0e4-8ff90e6937f0 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:49.188018 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152cb8ab-daef-43bf-a0e4-8ff90e6937f0 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:49.188561 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152cb8ab-daef-43bf-a0e4-8ff90e6937f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:49.207360 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-152cb8ab-daef-43bf-a0e4-8ff90e6937f0 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:46:49.208592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-152cb8ab-daef-43bf-a0e4-8ff90e6937f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:49.217164 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1197/2396] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:46:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:46:49.221580 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674fd6a3-3261-45f8-ab98-250f55d282c5 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:49.232118 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674fd6a3-3261-45f8-ab98-250f55d282c5 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:49.237314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a759cf-ba6a-4ea6-bfc5-e6da9321e7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:49.237990 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a759cf-ba6a-4ea6-bfc5-e6da9321e7bb admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:49.238273 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a759cf-ba6a-4ea6-bfc5-e6da9321e7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:49.238543 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a759cf-ba6a-4ea6-bfc5-e6da9321e7bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:49.239327 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a759cf-ba6a-4ea6-bfc5-e6da9321e7bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:46:49.239629 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a759cf-ba6a-4ea6-bfc5-e6da9321e7bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:49.241235 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a759cf-ba6a-4ea6-bfc5-e6da9321e7bb admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:49.253769 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1200/2397] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:46:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:46:49.258237 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152cb8ab-daef-43bf-a0e4-8ff90e6937f0 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:49.267583 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152cb8ab-daef-43bf-a0e4-8ff90e6937f0 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:49.272308 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989c996-0647-4114-aa8e-07c63e340842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:49.272948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989c996-0647-4114-aa8e-07c63e340842 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:49.273233 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989c996-0647-4114-aa8e-07c63e340842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:49.273509 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989c996-0647-4114-aa8e-07c63e340842 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:49.274201 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2989c996-0647-4114-aa8e-07c63e340842 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:49.276140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2989c996-0647-4114-aa8e-07c63e340842 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:49.304448 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1198/2398] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:46:49 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:46:49.308677 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a759cf-ba6a-4ea6-bfc5-e6da9321e7bb admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:49.319236 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a759cf-ba6a-4ea6-bfc5-e6da9321e7bb admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:49.374463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ec109-ae79-40b4-b973-838a9c3416e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:49.375104 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ec109-ae79-40b4-b973-838a9c3416e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:49.375388 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ec109-ae79-40b4-b973-838a9c3416e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:49.375666 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ec109-ae79-40b4-b973-838a9c3416e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:49.376359 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ec109-ae79-40b4-b973-838a9c3416e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:49.378273 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ec109-ae79-40b4-b973-838a9c3416e6 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:49.397912 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1201/2399] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:46:49 2022] POST /identity/v3/users => generated 296 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:46:49.401965 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989c996-0647-4114-aa8e-07c63e340842 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:49.413139 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989c996-0647-4114-aa8e-07c63e340842 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:49.417882 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff1b8d-714e-4661-8e94-b6b9239f81ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:49.419140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff1b8d-714e-4661-8e94-b6b9239f81ad admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:49.419424 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff1b8d-714e-4661-8e94-b6b9239f81ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:49.419696 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff1b8d-714e-4661-8e94-b6b9239f81ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:49.420421 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff1b8d-714e-4661-8e94-b6b9239f81ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:49.422031 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff1b8d-714e-4661-8e94-b6b9239f81ad admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:49.433065 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1199/2400] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:46:49.437715 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ec109-ae79-40b4-b973-838a9c3416e6 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:49.447712 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ec109-ae79-40b4-b973-838a9c3416e6 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:49.452469 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45633afc-64a1-41ae-908c-c4a50a5b00fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:49.453818 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45633afc-64a1-41ae-908c-c4a50a5b00fa admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:49.453818 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45633afc-64a1-41ae-908c-c4a50a5b00fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:49.453818 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45633afc-64a1-41ae-908c-c4a50a5b00fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:49.454422 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45633afc-64a1-41ae-908c-c4a50a5b00fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:49.456091 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45633afc-64a1-41ae-908c-c4a50a5b00fa admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:49.467746 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1202/2401] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:46:49.471587 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff1b8d-714e-4661-8e94-b6b9239f81ad admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:49.482516 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff1b8d-714e-4661-8e94-b6b9239f81ad admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:49.487242 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750d5cb-2dd0-4cd6-a165-80d28ad4949a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:49.488121 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750d5cb-2dd0-4cd6-a165-80d28ad4949a admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:49.488404 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750d5cb-2dd0-4cd6-a165-80d28ad4949a admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:49.488677 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750d5cb-2dd0-4cd6-a165-80d28ad4949a admin admin] PATH_INFO: `/v3/projects/5d579424e3b54e5b9809de22c68a68c6/users/4d356361fc6f40e59010b263e8d1dffa/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:49.503478 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6750d5cb-2dd0-4cd6-a165-80d28ad4949a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d579424e3b54e5b9809de22c68a68c6, user_id=4d356361fc6f40e59010b263e8d1dffa, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:49.506125 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6750d5cb-2dd0-4cd6-a165-80d28ad4949a admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:49.519710 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1200/2402] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:46:49 2022] PUT /identity/v3/projects/5d579424e3b54e5b9809de22c68a68c6/users/4d356361fc6f40e59010b263e8d1dffa/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:46:49.530773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29865f93-3326-4bee-99ef-4bf8ca5fb247 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:49.531096 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29865f93-3326-4bee-99ef-4bf8ca5fb247 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:49.531383 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29865f93-3326-4bee-99ef-4bf8ca5fb247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:49.557160 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-29865f93-3326-4bee-99ef-4bf8ca5fb247 None None] MFA Rules not processed for user `4d356361fc6f40e59010b263e8d1dffa`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:46:49.558366 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-29865f93-3326-4bee-99ef-4bf8ca5fb247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:49.621601 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1203/2403] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:46:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:46:49.632180 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750d5cb-2dd0-4cd6-a165-80d28ad4949a admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:49.633869 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-6750d5cb-2dd0-4cd6-a165-80d28ad4949a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:49.656822 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:46:49.692671 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750d5cb-2dd0-4cd6-a165-80d28ad4949a admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:49.696349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d39808c-9066-4970-89f5-a97bc37ed9a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:49.696983 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39808c-9066-4970-89f5-a97bc37ed9a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:49.697268 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39808c-9066-4970-89f5-a97bc37ed9a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:49.697549 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39808c-9066-4970-89f5-a97bc37ed9a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:49.706782 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d39808c-9066-4970-89f5-a97bc37ed9a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:49.708299 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d39808c-9066-4970-89f5-a97bc37ed9a3 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:49.717883 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1201/2404] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:46:49 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:46:59.618575 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:46:59.713255 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=17 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:47:14.020962 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df965f-2905-489d-af55-b6d9214c6b4c None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:14.021449 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df965f-2905-489d-af55-b6d9214c6b4c None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:14.021547 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df965f-2905-489d-af55-b6d9214c6b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:14.074652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-52df965f-2905-489d-af55-b6d9214c6b4c None None] MFA Rules not processed for user `bf393488726e49338e70017f5cb7c4be`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:47:14.075954 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-52df965f-2905-489d-af55-b6d9214c6b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:14.140402 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d39808c-9066-4970-89f5-a97bc37ed9a3 service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:47:14.158401 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1204/2405] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Jun 15 14:47:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:47:14.215801 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d39808c-9066-4970-89f5-a97bc37ed9a3 service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:47:14.221571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99b752-3377-4698-83c7-0ddf04d6f2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:47:14.222551 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99b752-3377-4698-83c7-0ddf04d6f2dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:14.222904 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99b752-3377-4698-83c7-0ddf04d6f2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:14.223512 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99b752-3377-4698-83c7-0ddf04d6f2dc admin admin] PATH_INFO: `/v3/users/4d356361fc6f40e59010b263e8d1dffa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:14.224849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99b752-3377-4698-83c7-0ddf04d6f2dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d356361fc6f40e59010b263e8d1dffa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:47:14.226789 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99b752-3377-4698-83c7-0ddf04d6f2dc admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:47:14.263199 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df965f-2905-489d-af55-b6d9214c6b4c None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:47:14.264922 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-52df965f-2905-489d-af55-b6d9214c6b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:14.287516 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-bd99b752-3377-4698-83c7-0ddf04d6f2dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d356361fc6f40e59010b263e8d1dffa'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:47:14.288287 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-bd99b752-3377-4698-83c7-0ddf04d6f2dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d356361fc6f40e59010b263e8d1dffa'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:47:14.307212 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-bd99b752-3377-4698-83c7-0ddf04d6f2dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d356361fc6f40e59010b263e8d1dffa'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:47:14.316682 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-bd99b752-3377-4698-83c7-0ddf04d6f2dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d356361fc6f40e59010b263e8d1dffa'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:47:14.325202 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-bd99b752-3377-4698-83c7-0ddf04d6f2dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d356361fc6f40e59010b263e8d1dffa'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:47:14.335077 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1202/2406] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:47:14 2022] DELETE /identity/v3/users/4d356361fc6f40e59010b263e8d1dffa => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:47:14.338715 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba348ad-76a3-496a-be0b-45ee4bbd8b28 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:14.339151 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba348ad-76a3-496a-be0b-45ee4bbd8b28 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:14.339617 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba348ad-76a3-496a-be0b-45ee4bbd8b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:14.367060 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-fba348ad-76a3-496a-be0b-45ee4bbd8b28 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:47:14.379090 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-fba348ad-76a3-496a-be0b-45ee4bbd8b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:14.399864 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df965f-2905-489d-af55-b6d9214c6b4c None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:47:14.436599 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1203/2407] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:47:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:47:14.446682 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba348ad-76a3-496a-be0b-45ee4bbd8b28 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:47:14.448883 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-fba348ad-76a3-496a-be0b-45ee4bbd8b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:14.460604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:47:14.473172 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a610d91-87c5-4abb-a10d-6893b3e3f14f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '641a81b3140f47119b5c2323a97a6681', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:47:14.474028 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a610d91-87c5-4abb-a10d-6893b3e3f14f service swift] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:14.474348 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a610d91-87c5-4abb-a10d-6893b3e3f14f service swift] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:14.474909 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a610d91-87c5-4abb-a10d-6893b3e3f14f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:14.477533 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:47:14.478475 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5a610d91-87c5-4abb-a10d-6893b3e3f14f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:14.531667 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba348ad-76a3-496a-be0b-45ee4bbd8b28 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:47:14.535674 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b581e355-d2ea-4998-9331-51f97670f22f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:47:14.536336 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581e355-d2ea-4998-9331-51f97670f22f service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:14.536621 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581e355-d2ea-4998-9331-51f97670f22f service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:14.536906 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581e355-d2ea-4998-9331-51f97670f22f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:14.549023 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b581e355-d2ea-4998-9331-51f97670f22f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:47:14.550613 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b581e355-d2ea-4998-9331-51f97670f22f service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:47:14.561515 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a610d91-87c5-4abb-a10d-6893b3e3f14f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:47:14.562828 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1204/2408] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:47:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:47:14.563417 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a610d91-87c5-4abb-a10d-6893b3e3f14f service swift] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:47:14.574807 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1205/2409] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Jun 15 14:47:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:47:14.933462 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b581e355-d2ea-4998-9331-51f97670f22f service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:47:14.934651 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b581e355-d2ea-4998-9331-51f97670f22f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:14.965849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b581e355-d2ea-4998-9331-51f97670f22f service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:47:14.975464 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e626efad-50ad-466d-8790-cbcf1efed06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:47:14.976866 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626efad-50ad-466d-8790-cbcf1efed06f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:14.977157 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626efad-50ad-466d-8790-cbcf1efed06f admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:14.977439 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626efad-50ad-466d-8790-cbcf1efed06f admin admin] PATH_INFO: `/v3/projects/5d579424e3b54e5b9809de22c68a68c6` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:14.980067 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e626efad-50ad-466d-8790-cbcf1efed06f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d579424e3b54e5b9809de22c68a68c6)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:47:14.982023 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e626efad-50ad-466d-8790-cbcf1efed06f admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:47:15.054222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e626efad-50ad-466d-8790-cbcf1efed06f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d579424e3b54e5b9809de22c68a68c6', 'request_id': 'req-e626efad-50ad-466d-8790-cbcf1efed06f'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:47:15.066109 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e626efad-50ad-466d-8790-cbcf1efed06f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d579424e3b54e5b9809de22c68a68c6', 'request_id': 'req-e626efad-50ad-466d-8790-cbcf1efed06f'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:47:15.078022 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e626efad-50ad-466d-8790-cbcf1efed06f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d579424e3b54e5b9809de22c68a68c6', 'request_id': 'req-e626efad-50ad-466d-8790-cbcf1efed06f'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:47:15.097836 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e626efad-50ad-466d-8790-cbcf1efed06f admin admin] The token cache is being invalidate because project 5d579424e3b54e5b9809de22c68a68c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:47:15.098460 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e626efad-50ad-466d-8790-cbcf1efed06f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:47:15.100263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e626efad-50ad-466d-8790-cbcf1efed06f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:47:15.102580 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1205/2410] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:47:14 2022] DELETE /identity/v3/projects/5d579424e3b54e5b9809de22c68a68c6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:47:24.569321 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=29 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:47:25.089142 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=44,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:47:28.395848 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ab3b9-ae86-484f-bc53-70f3ac618895 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:28.396414 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ab3b9-ae86-484f-bc53-70f3ac618895 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:28.396414 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ab3b9-ae86-484f-bc53-70f3ac618895 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:28.397783 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1206/2411] 10.222.0.45 () {54 vars in 921 bytes} [Wed Jun 15 14:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:47:28.404654 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255253d3-5694-442f-a5ff-22212d56ff34 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:28.405197 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255253d3-5694-442f-a5ff-22212d56ff34 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:28.405341 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255253d3-5694-442f-a5ff-22212d56ff34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:28.468500 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-255253d3-5694-442f-a5ff-22212d56ff34 None None] MFA Rules not processed for user `f3185bd2458c4e22830975700888e9be`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:47:28.469767 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-255253d3-5694-442f-a5ff-22212d56ff34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:28.554768 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1206/2412] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Jun 15 14:47:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:47:28.568196 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ab3b9-ae86-484f-bc53-70f3ac618895 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:47:28.569874 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-3d9ab3b9-ae86-484f-bc53-70f3ac618895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:28.653547 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ab3b9-ae86-484f-bc53-70f3ac618895 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:47:28.657882 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54555142-987c-4ba1-96c2-7a989959cd92 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f10214b2bb4eafb3dff7adf71b9bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:47:28.658531 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54555142-987c-4ba1-96c2-7a989959cd92 service glance] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:28.658814 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54555142-987c-4ba1-96c2-7a989959cd92 service glance] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:28.659124 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54555142-987c-4ba1-96c2-7a989959cd92 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:28.676030 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54555142-987c-4ba1-96c2-7a989959cd92 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:47:28.677521 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54555142-987c-4ba1-96c2-7a989959cd92 service glance] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:47:28.689360 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1207/2413] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:47:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:47:28.761228 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49206f-8c81-44d9-8d1b-646dfd3389fd None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:28.761592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49206f-8c81-44d9-8d1b-646dfd3389fd None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:28.761878 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49206f-8c81-44d9-8d1b-646dfd3389fd None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:28.763345 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1207/2414] 10.222.0.45 () {54 vars in 921 bytes} [Wed Jun 15 14:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:47:28.769036 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0f2c3-4d78-4cee-8425-050dafd27925 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:28.769289 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0f2c3-4d78-4cee-8425-050dafd27925 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:28.769569 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0f2c3-4d78-4cee-8425-050dafd27925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:28.789365 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-3bc0f2c3-4d78-4cee-8425-050dafd27925 None None] MFA Rules not processed for user `975ae2790ac849ff88e991231671f510`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:47:28.790578 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-3bc0f2c3-4d78-4cee-8425-050dafd27925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:28.846197 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1208/2415] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Jun 15 14:47:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:47:28.857839 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb49206f-8c81-44d9-8d1b-646dfd3389fd None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:47:28.860127 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-fb49206f-8c81-44d9-8d1b-646dfd3389fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:28.874426 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb49206f-8c81-44d9-8d1b-646dfd3389fd None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:47:28.878246 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d399d851-c936-49f3-8262-607ab0dee78f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:47:28.878889 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399d851-c936-49f3-8262-607ab0dee78f service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:28.879203 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399d851-c936-49f3-8262-607ab0dee78f service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:28.879483 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399d851-c936-49f3-8262-607ab0dee78f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:28.887130 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d399d851-c936-49f3-8262-607ab0dee78f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:47:28.888562 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d399d851-c936-49f3-8262-607ab0dee78f service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:47:28.897660 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1208/2416] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:47:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:47:28.986709 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258f1cd-42ae-4c46-ac2b-e1098d4c666b None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:28.987013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258f1cd-42ae-4c46-ac2b-e1098d4c666b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:28.987318 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258f1cd-42ae-4c46-ac2b-e1098d4c666b None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:28.988707 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1209/2417] 10.222.0.45 () {54 vars in 921 bytes} [Wed Jun 15 14:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:47:28.994443 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6ffc4-08b7-464b-b5ec-6938c9606f22 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:28.994752 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6ffc4-08b7-464b-b5ec-6938c9606f22 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:28.995054 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6ffc4-08b7-464b-b5ec-6938c9606f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:29.018277 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-62a6ffc4-08b7-464b-b5ec-6938c9606f22 None None] MFA Rules not processed for user `f3185bd2458c4e22830975700888e9be`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:47:29.019603 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-62a6ffc4-08b7-464b-b5ec-6938c9606f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:29.028657 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1209/2418] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Jun 15 14:47:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:47:29.040050 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258f1cd-42ae-4c46-ac2b-e1098d4c666b None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:47:29.051182 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258f1cd-42ae-4c46-ac2b-e1098d4c666b None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:47:29.054844 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68299d5a-d747-4dc7-8c73-410b557ee310 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:47:29.055524 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68299d5a-d747-4dc7-8c73-410b557ee310 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:29.055810 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68299d5a-d747-4dc7-8c73-410b557ee310 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:29.056088 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68299d5a-d747-4dc7-8c73-410b557ee310 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:29.069218 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68299d5a-d747-4dc7-8c73-410b557ee310 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:47:29.071041 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68299d5a-d747-4dc7-8c73-410b557ee310 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:47:29.083789 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1210/2419] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:47:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:47:31.905806 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e1380-4119-4c18-bed7-ea6873fe7306 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:31.906279 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e1380-4119-4c18-bed7-ea6873fe7306 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:31.906384 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e1380-4119-4c18-bed7-ea6873fe7306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:31.925438 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-853e1380-4119-4c18-bed7-ea6873fe7306 None None] MFA Rules not processed for user `bf393488726e49338e70017f5cb7c4be`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:47:31.926645 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-853e1380-4119-4c18-bed7-ea6873fe7306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:32.010939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1210/2420] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Jun 15 14:47:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:47:32.026427 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68299d5a-d747-4dc7-8c73-410b557ee310 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:47:32.027936 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-68299d5a-d747-4dc7-8c73-410b557ee310 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:32.110891 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68299d5a-d747-4dc7-8c73-410b557ee310 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:47:32.116262 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f981c-df8e-45e6-b748-da4f7ac46d45 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '641a81b3140f47119b5c2323a97a6681', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:47:32.116911 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f981c-df8e-45e6-b748-da4f7ac46d45 service swift] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:32.117208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f981c-df8e-45e6-b748-da4f7ac46d45 service swift] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:32.117463 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f981c-df8e-45e6-b748-da4f7ac46d45 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:32.127125 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f981c-df8e-45e6-b748-da4f7ac46d45 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:47:32.128729 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f981c-df8e-45e6-b748-da4f7ac46d45 service swift] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:47:32.137838 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1211/2421] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Jun 15 14:47:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:47:41.993731 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=71 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:47:42.134062 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=96 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:47:45.086088 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e1380-4119-4c18-bed7-ea6873fe7306 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:47:45.087516 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-853e1380-4119-4c18-bed7-ea6873fe7306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:45.099395 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e1380-4119-4c18-bed7-ea6873fe7306 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:47:45.102915 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287cc485-524f-48b5-a7b6-fd3de3d8a307 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:47:45.103579 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287cc485-524f-48b5-a7b6-fd3de3d8a307 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:45.103865 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287cc485-524f-48b5-a7b6-fd3de3d8a307 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:45.104143 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287cc485-524f-48b5-a7b6-fd3de3d8a307 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:45.105836 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-287cc485-524f-48b5-a7b6-fd3de3d8a307 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:45.118546 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287cc485-524f-48b5-a7b6-fd3de3d8a307 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:47:45.120004 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287cc485-524f-48b5-a7b6-fd3de3d8a307 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:47:45.130745 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1211/2422] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:47:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:47:46.313883 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f981c-df8e-45e6-b748-da4f7ac46d45 service swift] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:47:46.323526 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f981c-df8e-45e6-b748-da4f7ac46d45 service swift] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:47:46.330225 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd2fede-5696-42c8-8157-d6b6e7c9cb99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:47:46.331093 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2fede-5696-42c8-8157-d6b6e7c9cb99 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:46.331379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2fede-5696-42c8-8157-d6b6e7c9cb99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:46.331652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2fede-5696-42c8-8157-d6b6e7c9cb99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:46.333653 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-9cd2fede-5696-42c8-8157-d6b6e7c9cb99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:46.349880 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd2fede-5696-42c8-8157-d6b6e7c9cb99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:47:46.351331 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd2fede-5696-42c8-8157-d6b6e7c9cb99 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:47:46.362798 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1212/2423] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:47:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:47:55.125947 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=6 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:47:56.357312 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=6 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:47:56.644582 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287cc485-524f-48b5-a7b6-fd3de3d8a307 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:47:56.646078 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-287cc485-524f-48b5-a7b6-fd3de3d8a307 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:56.702890 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287cc485-524f-48b5-a7b6-fd3de3d8a307 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:47:56.707671 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34efc30c-8343-4c8d-b6c7-fed426b81c5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:47:56.708332 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34efc30c-8343-4c8d-b6c7-fed426b81c5b service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:56.708617 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34efc30c-8343-4c8d-b6c7-fed426b81c5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:56.708891 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34efc30c-8343-4c8d-b6c7-fed426b81c5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:56.710675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-34efc30c-8343-4c8d-b6c7-fed426b81c5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:56.769138 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34efc30c-8343-4c8d-b6c7-fed426b81c5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:47:56.770574 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34efc30c-8343-4c8d-b6c7-fed426b81c5b service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:47:56.779482 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1212/2424] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:47:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:47:59.222513 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd2fede-5696-42c8-8157-d6b6e7c9cb99 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:47:59.224233 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-9cd2fede-5696-42c8-8157-d6b6e7c9cb99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:59.238436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd2fede-5696-42c8-8157-d6b6e7c9cb99 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:47:59.244232 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae3d3b-f416-417e-b062-7b7aa120a319 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:47:59.244898 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae3d3b-f416-417e-b062-7b7aa120a319 service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:59.245183 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae3d3b-f416-417e-b062-7b7aa120a319 service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:59.245468 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae3d3b-f416-417e-b062-7b7aa120a319 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:59.247341 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-aeae3d3b-f416-417e-b062-7b7aa120a319 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:59.312042 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae3d3b-f416-417e-b062-7b7aa120a319 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:47:59.313789 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae3d3b-f416-417e-b062-7b7aa120a319 service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:47:59.323748 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1213/2425] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:47:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:48:02.627987 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd86468-2346-479c-a4ab-163c3a9203b5 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:02.628485 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd86468-2346-479c-a4ab-163c3a9203b5 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:02.628591 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd86468-2346-479c-a4ab-163c3a9203b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:02.646207 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-2dd86468-2346-479c-a4ab-163c3a9203b5 None None] MFA Rules not processed for user `bf393488726e49338e70017f5cb7c4be`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:02.647444 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-2dd86468-2346-479c-a4ab-163c3a9203b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:02.654471 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1213/2426] 10.222.0.45 () {58 vars in 967 bytes} [Wed Jun 15 14:48:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:48:02.666983 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae3d3b-f416-417e-b062-7b7aa120a319 service nova] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:02.678117 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae3d3b-f416-417e-b062-7b7aa120a319 service nova] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:02.683335 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5389e50-fec4-4278-a08c-2fa5327923b7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '641a81b3140f47119b5c2323a97a6681', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:02.684012 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5389e50-fec4-4278-a08c-2fa5327923b7 service swift] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:02.684299 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5389e50-fec4-4278-a08c-2fa5327923b7 service swift] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:02.684577 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5389e50-fec4-4278-a08c-2fa5327923b7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:02.693254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5389e50-fec4-4278-a08c-2fa5327923b7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:02.694832 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5389e50-fec4-4278-a08c-2fa5327923b7 service swift] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:02.702844 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1214/2427] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Jun 15 14:48:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:48:11.269649 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd86468-2346-479c-a4ab-163c3a9203b5 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:11.283069 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd86468-2346-479c-a4ab-163c3a9203b5 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:11.287356 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ba419-9cf6-4659-86a2-da92f8abf290 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:11.288253 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ba419-9cf6-4659-86a2-da92f8abf290 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:11.288542 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ba419-9cf6-4659-86a2-da92f8abf290 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:11.288826 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ba419-9cf6-4659-86a2-da92f8abf290 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:11.290951 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d79ba419-9cf6-4659-86a2-da92f8abf290 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:11.302029 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ba419-9cf6-4659-86a2-da92f8abf290 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:11.303515 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ba419-9cf6-4659-86a2-da92f8abf290 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:11.311944 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1214/2428] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:48:12.699715 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=34 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:48:13.910543 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c18b47-3af5-451b-baea-1f9871bf0790 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:13.911194 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c18b47-3af5-451b-baea-1f9871bf0790 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:13.911194 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c18b47-3af5-451b-baea-1f9871bf0790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:13.928135 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-95c18b47-3af5-451b-baea-1f9871bf0790 None None] MFA Rules not processed for user `c9a3a2e49fee4ea08ca790006d0b29e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:13.929333 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-95c18b47-3af5-451b-baea-1f9871bf0790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:13.936287 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1215/2429] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:48:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:48:20.906026 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ba419-9cf6-4659-86a2-da92f8abf290 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:20.919448 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ba419-9cf6-4659-86a2-da92f8abf290 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:20.926006 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044943bf-2e66-4b5a-913a-21aa349c0d41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:20.926658 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044943bf-2e66-4b5a-913a-21aa349c0d41 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:20.926946 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044943bf-2e66-4b5a-913a-21aa349c0d41 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:20.927259 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044943bf-2e66-4b5a-913a-21aa349c0d41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:20.929040 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-044943bf-2e66-4b5a-913a-21aa349c0d41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:20.942208 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044943bf-2e66-4b5a-913a-21aa349c0d41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:20.943835 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044943bf-2e66-4b5a-913a-21aa349c0d41 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:20.955131 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1215/2430] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:48:23.923739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:48:24.083005 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c18b47-3af5-451b-baea-1f9871bf0790 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:24.084456 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-95c18b47-3af5-451b-baea-1f9871bf0790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:24.100831 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c18b47-3af5-451b-baea-1f9871bf0790 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:24.106304 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8970eae-cc24-4a29-b773-a4029e0a84e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:24.106975 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8970eae-cc24-4a29-b773-a4029e0a84e1 service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:24.107287 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8970eae-cc24-4a29-b773-a4029e0a84e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:24.107568 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8970eae-cc24-4a29-b773-a4029e0a84e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:24.109338 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-b8970eae-cc24-4a29-b773-a4029e0a84e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:24.168362 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8970eae-cc24-4a29-b773-a4029e0a84e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:24.169978 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8970eae-cc24-4a29-b773-a4029e0a84e1 service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:24.180822 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1216/2431] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:48:30.950274 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=63 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:48:34.176181 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=28 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:49:17.326383 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044943bf-2e66-4b5a-913a-21aa349c0d41 service nova] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:17.338070 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044943bf-2e66-4b5a-913a-21aa349c0d41 service nova] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:17.343889 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec4e24-129f-4a30-bba1-dab2180c1993 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:17.354628 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec4e24-129f-4a30-bba1-dab2180c1993 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:17.355025 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec4e24-129f-4a30-bba1-dab2180c1993 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:17.355244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec4e24-129f-4a30-bba1-dab2180c1993 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:17.357852 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-0dec4e24-129f-4a30-bba1-dab2180c1993 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:17.372618 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec4e24-129f-4a30-bba1-dab2180c1993 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:17.374222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec4e24-129f-4a30-bba1-dab2180c1993 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:17.410650 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1216/2432] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:49:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:49:21.102406 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23012c7-ef11-44e4-9656-a4778fc880d0 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:21.102976 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23012c7-ef11-44e4-9656-a4778fc880d0 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:21.103146 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23012c7-ef11-44e4-9656-a4778fc880d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:21.120417 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-b23012c7-ef11-44e4-9656-a4778fc880d0 None None] MFA Rules not processed for user `25ba370b8daf49e48514cd49c2851215`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:49:21.121645 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-b23012c7-ef11-44e4-9656-a4778fc880d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:21.129314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1217/2433] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:49:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:49:21.457721 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec4e24-129f-4a30-bba1-dab2180c1993 service nova] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:21.466838 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec4e24-129f-4a30-bba1-dab2180c1993 service nova] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:21.470374 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dcf01a-5478-4016-b653-0ab2bc79bba9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:21.471212 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dcf01a-5478-4016-b653-0ab2bc79bba9 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:21.471500 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dcf01a-5478-4016-b653-0ab2bc79bba9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:21.471778 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dcf01a-5478-4016-b653-0ab2bc79bba9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:21.473793 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b4dcf01a-5478-4016-b653-0ab2bc79bba9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:21.483647 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dcf01a-5478-4016-b653-0ab2bc79bba9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:21.485062 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dcf01a-5478-4016-b653-0ab2bc79bba9 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:21.493435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1217/2434] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:49:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:49:21.810064 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23012c7-ef11-44e4-9656-a4778fc880d0 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:21.811658 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-b23012c7-ef11-44e4-9656-a4778fc880d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:21.888107 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23012c7-ef11-44e4-9656-a4778fc880d0 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:21.893202 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c0204b-9ec1-47e6-869a-f8628e4e4470 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4128111a9aa450abb2146a5d9926dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:21.893857 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0204b-9ec1-47e6-869a-f8628e4e4470 service placement] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:21.894146 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0204b-9ec1-47e6-869a-f8628e4e4470 service placement] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:21.894428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0204b-9ec1-47e6-869a-f8628e4e4470 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:21.896234 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d4c0204b-9ec1-47e6-869a-f8628e4e4470 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:21.909125 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c0204b-9ec1-47e6-869a-f8628e4e4470 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:21.911001 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c0204b-9ec1-47e6-869a-f8628e4e4470 service placement] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:21.922602 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1218/2435] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:49:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:49:31.490000 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=12 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:49:31.821360 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dcf01a-5478-4016-b653-0ab2bc79bba9 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:31.838974 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dcf01a-5478-4016-b653-0ab2bc79bba9 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:31.844655 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08f892-9ad9-4ccc-b25b-bb604a56e48a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:31.845567 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08f892-9ad9-4ccc-b25b-bb604a56e48a service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:31.845567 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08f892-9ad9-4ccc-b25b-bb604a56e48a service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:31.845857 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08f892-9ad9-4ccc-b25b-bb604a56e48a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:31.847862 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-dd08f892-9ad9-4ccc-b25b-bb604a56e48a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:31.860924 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd08f892-9ad9-4ccc-b25b-bb604a56e48a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:31.862791 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd08f892-9ad9-4ccc-b25b-bb604a56e48a service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:31.874666 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1218/2436] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:49:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:49:31.917473 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=35 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:49:41.869303 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=6 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:49:53.080706 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51eef2-5c42-47fe-b0fa-fd37adc26c74 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:53.081240 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51eef2-5c42-47fe-b0fa-fd37adc26c74 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:53.081240 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51eef2-5c42-47fe-b0fa-fd37adc26c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:53.099522 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-3a51eef2-5c42-47fe-b0fa-fd37adc26c74 None None] MFA Rules not processed for user `1efb0951ba1144d58698e02e0e5e9416`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:49:53.100757 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-3a51eef2-5c42-47fe-b0fa-fd37adc26c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:53.107938 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1219/2437] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:49:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:49:53.408760 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08f892-9ad9-4ccc-b25b-bb604a56e48a service nova] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:53.418418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08f892-9ad9-4ccc-b25b-bb604a56e48a service nova] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:53.421979 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce2ccc-8b13-43a8-892e-478fed904639 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:53.422607 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce2ccc-8b13-43a8-892e-478fed904639 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:53.422890 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce2ccc-8b13-43a8-892e-478fed904639 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:53.423201 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce2ccc-8b13-43a8-892e-478fed904639 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:53.425043 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-40ce2ccc-8b13-43a8-892e-478fed904639 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:53.434919 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce2ccc-8b13-43a8-892e-478fed904639 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:53.436370 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce2ccc-8b13-43a8-892e-478fed904639 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:53.444937 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1219/2438] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:49:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:49:53.629661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51eef2-5c42-47fe-b0fa-fd37adc26c74 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:53.640020 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51eef2-5c42-47fe-b0fa-fd37adc26c74 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:53.643952 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814b511-e9f9-4f47-9aed-8fb346366dc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:53.644537 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814b511-e9f9-4f47-9aed-8fb346366dc4 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:53.644936 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814b511-e9f9-4f47-9aed-8fb346366dc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:53.645215 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814b511-e9f9-4f47-9aed-8fb346366dc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:53.646983 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f814b511-e9f9-4f47-9aed-8fb346366dc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:53.719493 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814b511-e9f9-4f47-9aed-8fb346366dc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:53.720981 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814b511-e9f9-4f47-9aed-8fb346366dc4 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:53.733877 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1220/2439] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:49:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:49:58.068344 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce2ccc-8b13-43a8-892e-478fed904639 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:58.081324 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce2ccc-8b13-43a8-892e-478fed904639 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:58.087156 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d83af-55f0-42b3-a8dc-2b09f82ee579 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4128111a9aa450abb2146a5d9926dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:58.087926 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d83af-55f0-42b3-a8dc-2b09f82ee579 service placement] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:58.088509 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d83af-55f0-42b3-a8dc-2b09f82ee579 service placement] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:58.089015 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d83af-55f0-42b3-a8dc-2b09f82ee579 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:58.091643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c90d83af-55f0-42b3-a8dc-2b09f82ee579 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:58.113261 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90d83af-55f0-42b3-a8dc-2b09f82ee579 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:58.115283 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90d83af-55f0-42b3-a8dc-2b09f82ee579 service placement] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:58.121004 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814b511-e9f9-4f47-9aed-8fb346366dc4 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:58.122847 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f814b511-e9f9-4f47-9aed-8fb346366dc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:58.128594 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1220/2440] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:49:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:49:58.137609 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814b511-e9f9-4f47-9aed-8fb346366dc4 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:58.142719 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031b7a7-43d0-483a-9f50-95fed9f52508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:58.143640 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031b7a7-43d0-483a-9f50-95fed9f52508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:58.143920 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031b7a7-43d0-483a-9f50-95fed9f52508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:58.144242 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031b7a7-43d0-483a-9f50-95fed9f52508 admin admin] PATH_INFO: `/v3/users/c9a3a2e49fee4ea08ca790006d0b29e4` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:58.145551 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3031b7a7-43d0-483a-9f50-95fed9f52508 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9a3a2e49fee4ea08ca790006d0b29e4)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:58.147595 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3031b7a7-43d0-483a-9f50-95fed9f52508 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:58.212964 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3031b7a7-43d0-483a-9f50-95fed9f52508 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9a3a2e49fee4ea08ca790006d0b29e4'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:49:58.213625 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3031b7a7-43d0-483a-9f50-95fed9f52508 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9a3a2e49fee4ea08ca790006d0b29e4'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:49:58.226078 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3031b7a7-43d0-483a-9f50-95fed9f52508 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9a3a2e49fee4ea08ca790006d0b29e4'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:49:58.226779 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3031b7a7-43d0-483a-9f50-95fed9f52508 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9a3a2e49fee4ea08ca790006d0b29e4'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:49:58.234628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3031b7a7-43d0-483a-9f50-95fed9f52508 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9a3a2e49fee4ea08ca790006d0b29e4'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:49:58.247878 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1221/2441] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:49:58 2022] DELETE /identity/v3/users/c9a3a2e49fee4ea08ca790006d0b29e4 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:49:58.445054 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d83af-55f0-42b3-a8dc-2b09f82ee579 service placement] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:58.455978 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d83af-55f0-42b3-a8dc-2b09f82ee579 service placement] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:58.514880 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc63a1-0406-4b1b-b795-6996d7baf969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:58.515779 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc63a1-0406-4b1b-b795-6996d7baf969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:58.516067 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc63a1-0406-4b1b-b795-6996d7baf969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:58.516373 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc63a1-0406-4b1b-b795-6996d7baf969 admin admin] PATH_INFO: `/v3/projects/a243ba2c3f774c10b77a052075497b92` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:58.517590 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edc63a1-0406-4b1b-b795-6996d7baf969 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a243ba2c3f774c10b77a052075497b92)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:58.519756 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edc63a1-0406-4b1b-b795-6996d7baf969 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:58.570701 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-5edc63a1-0406-4b1b-b795-6996d7baf969 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a243ba2c3f774c10b77a052075497b92', 'request_id': 'req-5edc63a1-0406-4b1b-b795-6996d7baf969'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:49:58.580900 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-5edc63a1-0406-4b1b-b795-6996d7baf969 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a243ba2c3f774c10b77a052075497b92', 'request_id': 'req-5edc63a1-0406-4b1b-b795-6996d7baf969'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:49:58.593379 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-5edc63a1-0406-4b1b-b795-6996d7baf969 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a243ba2c3f774c10b77a052075497b92', 'request_id': 'req-5edc63a1-0406-4b1b-b795-6996d7baf969'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:49:58.611539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-5edc63a1-0406-4b1b-b795-6996d7baf969 admin admin] The token cache is being invalidate because project a243ba2c3f774c10b77a052075497b92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:49:58.612595 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-5edc63a1-0406-4b1b-b795-6996d7baf969 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:49:58.613667 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-5edc63a1-0406-4b1b-b795-6996d7baf969 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:49:58.616370 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1221/2442] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:49:58 2022] DELETE /identity/v3/projects/a243ba2c3f774c10b77a052075497b92 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:49:58.624467 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86165249-d4aa-4bbb-85f9-2c3da45339fb None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:58.624964 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86165249-d4aa-4bbb-85f9-2c3da45339fb None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:58.625450 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86165249-d4aa-4bbb-85f9-2c3da45339fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:58.648485 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-86165249-d4aa-4bbb-85f9-2c3da45339fb None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:49:58.649710 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-86165249-d4aa-4bbb-85f9-2c3da45339fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:58.706419 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:49:58.727209 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1222/2443] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:49:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:49:58.730822 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22988e27-f4c8-40a6-a252-d831d55dbd2d None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:58.731586 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22988e27-f4c8-40a6-a252-d831d55dbd2d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:58.731980 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22988e27-f4c8-40a6-a252-d831d55dbd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:58.754451 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-22988e27-f4c8-40a6-a252-d831d55dbd2d None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:49:58.755976 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-22988e27-f4c8-40a6-a252-d831d55dbd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:58.764393 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1222/2444] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:49:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:49:58.769314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86165249-d4aa-4bbb-85f9-2c3da45339fb None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:58.779852 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86165249-d4aa-4bbb-85f9-2c3da45339fb None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:58.787435 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a99299-ac73-4ff0-91f6-e21b0f8385c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:58.788217 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a99299-ac73-4ff0-91f6-e21b0f8385c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:58.788506 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a99299-ac73-4ff0-91f6-e21b0f8385c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:58.788791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a99299-ac73-4ff0-91f6-e21b0f8385c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:58.789540 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a99299-ac73-4ff0-91f6-e21b0f8385c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:49:58.789885 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a99299-ac73-4ff0-91f6-e21b0f8385c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:58.791602 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a99299-ac73-4ff0-91f6-e21b0f8385c7 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:58.804794 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1223/2445] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:49:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:49:58.809400 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22988e27-f4c8-40a6-a252-d831d55dbd2d None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:58.820213 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22988e27-f4c8-40a6-a252-d831d55dbd2d None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:58.825026 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a362782a-c06b-4c08-8809-5c7ff05de1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:58.827686 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362782a-c06b-4c08-8809-5c7ff05de1ff admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:58.827686 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362782a-c06b-4c08-8809-5c7ff05de1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:58.827686 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362782a-c06b-4c08-8809-5c7ff05de1ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:58.827686 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a362782a-c06b-4c08-8809-5c7ff05de1ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:58.829066 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a362782a-c06b-4c08-8809-5c7ff05de1ff admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:58.873014 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1223/2446] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:49:58 2022] POST /identity/v3/projects => generated 387 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:49:58.877424 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a99299-ac73-4ff0-91f6-e21b0f8385c7 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:58.888559 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a99299-ac73-4ff0-91f6-e21b0f8385c7 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:58.966058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e58a341-d9cd-484f-acf4-013ce6e90e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:58.966685 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58a341-d9cd-484f-acf4-013ce6e90e1d admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:58.966972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58a341-d9cd-484f-acf4-013ce6e90e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:58.967282 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58a341-d9cd-484f-acf4-013ce6e90e1d admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:58.967983 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e58a341-d9cd-484f-acf4-013ce6e90e1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:58.970162 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e58a341-d9cd-484f-acf4-013ce6e90e1d admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:58.998945 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1224/2447] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:49:58 2022] POST /identity/v3/users => generated 294 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:49:59.001345 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a362782a-c06b-4c08-8809-5c7ff05de1ff admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:59.016382 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a362782a-c06b-4c08-8809-5c7ff05de1ff admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:59.022290 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886bea3-49c6-4eb5-8c5a-c71629df0df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:59.023578 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886bea3-49c6-4eb5-8c5a-c71629df0df8 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:59.023956 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886bea3-49c6-4eb5-8c5a-c71629df0df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:59.024503 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886bea3-49c6-4eb5-8c5a-c71629df0df8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:59.026538 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f886bea3-49c6-4eb5-8c5a-c71629df0df8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:59.028186 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f886bea3-49c6-4eb5-8c5a-c71629df0df8 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:59.040318 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1224/2448] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:49:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:49:59.044630 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e58a341-d9cd-484f-acf4-013ce6e90e1d admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:59.055531 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e58a341-d9cd-484f-acf4-013ce6e90e1d admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:59.060193 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5623599d-0adf-4a43-b754-df40e0c5836d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:59.060860 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623599d-0adf-4a43-b754-df40e0c5836d admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:59.061146 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623599d-0adf-4a43-b754-df40e0c5836d admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:59.061425 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623599d-0adf-4a43-b754-df40e0c5836d admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:59.062156 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5623599d-0adf-4a43-b754-df40e0c5836d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:59.063819 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5623599d-0adf-4a43-b754-df40e0c5836d admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:59.075276 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1225/2449] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:49:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:49:59.080800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886bea3-49c6-4eb5-8c5a-c71629df0df8 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:59.090936 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886bea3-49c6-4eb5-8c5a-c71629df0df8 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:59.098154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e335e-7256-4b21-9f2e-3c03c731483a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:59.099405 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e335e-7256-4b21-9f2e-3c03c731483a admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:59.099867 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e335e-7256-4b21-9f2e-3c03c731483a admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:59.100254 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e335e-7256-4b21-9f2e-3c03c731483a admin admin] PATH_INFO: `/v3/projects/cef572b6559d4e3a98082cd4ef500b2c/users/e8e1e57c0f68419dad301ab4f30ba7a3/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:59.113067 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:49:59.116388 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e335e-7256-4b21-9f2e-3c03c731483a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cef572b6559d4e3a98082cd4ef500b2c, user_id=e8e1e57c0f68419dad301ab4f30ba7a3, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:59.121267 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e335e-7256-4b21-9f2e-3c03c731483a admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:59.141489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1225/2450] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:49:59 2022] PUT /identity/v3/projects/cef572b6559d4e3a98082cd4ef500b2c/users/e8e1e57c0f68419dad301ab4f30ba7a3/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:49:59.152653 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a87200-1754-4773-9f5d-81819d4dae63 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:59.153201 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a87200-1754-4773-9f5d-81819d4dae63 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:59.153201 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a87200-1754-4773-9f5d-81819d4dae63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:59.182617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-95a87200-1754-4773-9f5d-81819d4dae63 None None] MFA Rules not processed for user `e8e1e57c0f68419dad301ab4f30ba7a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:49:59.183816 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-95a87200-1754-4773-9f5d-81819d4dae63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:59.258406 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1226/2451] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:49:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:49:59.266908 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e335e-7256-4b21-9f2e-3c03c731483a admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:59.337820 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e335e-7256-4b21-9f2e-3c03c731483a admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:59.342542 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bfc7c5-c17f-4231-b554-513b5fd6e2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:59.343233 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfc7c5-c17f-4231-b554-513b5fd6e2d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:59.343522 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfc7c5-c17f-4231-b554-513b5fd6e2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:59.343803 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfc7c5-c17f-4231-b554-513b5fd6e2d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:59.344531 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bfc7c5-c17f-4231-b554-513b5fd6e2d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:59.346442 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bfc7c5-c17f-4231-b554-513b5fd6e2d6 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:59.372733 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1226/2452] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:49:59 2022] POST /identity/v3/projects => generated 387 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:49:59.377555 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a87200-1754-4773-9f5d-81819d4dae63 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:59.388907 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a87200-1754-4773-9f5d-81819d4dae63 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:59.441092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:49:59.455940 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e7bcea-1c07-4f26-961b-2460d48da06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:59.456628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7bcea-1c07-4f26-961b-2460d48da06d admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:59.456912 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7bcea-1c07-4f26-961b-2460d48da06d admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:59.457188 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7bcea-1c07-4f26-961b-2460d48da06d admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:59.457899 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e7bcea-1c07-4f26-961b-2460d48da06d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:59.459880 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e7bcea-1c07-4f26-961b-2460d48da06d admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:59.480756 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1227/2453] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:49:59 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:49:59.485053 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bfc7c5-c17f-4231-b554-513b5fd6e2d6 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:59.495806 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bfc7c5-c17f-4231-b554-513b5fd6e2d6 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:59.503232 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c3f88-6f7d-4b71-bd55-277677cb8340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:59.503913 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c3f88-6f7d-4b71-bd55-277677cb8340 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:59.504199 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c3f88-6f7d-4b71-bd55-277677cb8340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:59.504479 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c3f88-6f7d-4b71-bd55-277677cb8340 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:59.505218 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5c3f88-6f7d-4b71-bd55-277677cb8340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:59.506843 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5c3f88-6f7d-4b71-bd55-277677cb8340 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:59.518658 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1227/2454] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:49:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:49:59.522905 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e7bcea-1c07-4f26-961b-2460d48da06d admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:59.534669 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e7bcea-1c07-4f26-961b-2460d48da06d admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:59.539828 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60065d-7034-4ed2-af93-8ea64f996a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:59.540493 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60065d-7034-4ed2-af93-8ea64f996a10 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:59.540778 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60065d-7034-4ed2-af93-8ea64f996a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:59.541055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60065d-7034-4ed2-af93-8ea64f996a10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:59.541865 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad60065d-7034-4ed2-af93-8ea64f996a10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:59.543517 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad60065d-7034-4ed2-af93-8ea64f996a10 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:59.556468 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1228/2455] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:49:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:49:59.562469 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c3f88-6f7d-4b71-bd55-277677cb8340 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:59.575763 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c3f88-6f7d-4b71-bd55-277677cb8340 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:59.582080 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d33659-2041-444a-996f-5b32bdef8252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:59.583039 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d33659-2041-444a-996f-5b32bdef8252 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:59.583325 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d33659-2041-444a-996f-5b32bdef8252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:59.583602 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d33659-2041-444a-996f-5b32bdef8252 admin admin] PATH_INFO: `/v3/projects/0238208299734f04826c1590b12e1c0f/users/b03368f828b2424b90e7e72c84070576/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:59.601080 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d33659-2041-444a-996f-5b32bdef8252 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0238208299734f04826c1590b12e1c0f, user_id=b03368f828b2424b90e7e72c84070576, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:59.603999 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d33659-2041-444a-996f-5b32bdef8252 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:59.618505 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1228/2456] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:49:59 2022] PUT /identity/v3/projects/0238208299734f04826c1590b12e1c0f/users/b03368f828b2424b90e7e72c84070576/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:49:59.630380 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66e734-95a9-449b-80e8-f6519d82cef2 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:59.631210 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66e734-95a9-449b-80e8-f6519d82cef2 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:59.631492 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66e734-95a9-449b-80e8-f6519d82cef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:59.657576 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-8b66e734-95a9-449b-80e8-f6519d82cef2 None None] MFA Rules not processed for user `b03368f828b2424b90e7e72c84070576`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:49:59.658788 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-8b66e734-95a9-449b-80e8-f6519d82cef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:59.723407 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1229/2457] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:49:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:49:59.727725 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d33659-2041-444a-996f-5b32bdef8252 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:59.793479 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d33659-2041-444a-996f-5b32bdef8252 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:59.798246 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe7a50c-3c55-40cd-a3a8-be1d7fee7f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:59.799456 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe7a50c-3c55-40cd-a3a8-be1d7fee7f4d admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:59.799743 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe7a50c-3c55-40cd-a3a8-be1d7fee7f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:59.800020 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe7a50c-3c55-40cd-a3a8-be1d7fee7f4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:59.800851 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe7a50c-3c55-40cd-a3a8-be1d7fee7f4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:59.802777 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe7a50c-3c55-40cd-a3a8-be1d7fee7f4d admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:59.835715 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1229/2458] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:49:59 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:49:59.836909 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66e734-95a9-449b-80e8-f6519d82cef2 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:59.850719 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66e734-95a9-449b-80e8-f6519d82cef2 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:59.910052 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50534e7b-9ea6-4192-89db-fc8869fd0a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:59.911019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50534e7b-9ea6-4192-89db-fc8869fd0a83 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:59.911305 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50534e7b-9ea6-4192-89db-fc8869fd0a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:59.911582 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50534e7b-9ea6-4192-89db-fc8869fd0a83 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:59.912292 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50534e7b-9ea6-4192-89db-fc8869fd0a83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:59.914220 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50534e7b-9ea6-4192-89db-fc8869fd0a83 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:59.936122 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1230/2459] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:49:59 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:49:59.940210 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe7a50c-3c55-40cd-a3a8-be1d7fee7f4d admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:59.950706 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe7a50c-3c55-40cd-a3a8-be1d7fee7f4d admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:59.955418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ee7cf-fbe8-4844-b547-0dd5390bd629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:59.956074 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ee7cf-fbe8-4844-b547-0dd5390bd629 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:59.956361 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ee7cf-fbe8-4844-b547-0dd5390bd629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:59.956633 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ee7cf-fbe8-4844-b547-0dd5390bd629 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:59.957351 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8ee7cf-fbe8-4844-b547-0dd5390bd629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:59.958938 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8ee7cf-fbe8-4844-b547-0dd5390bd629 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:59.969453 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1230/2460] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:49:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:49:59.974160 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50534e7b-9ea6-4192-89db-fc8869fd0a83 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:59.984076 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50534e7b-9ea6-4192-89db-fc8869fd0a83 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:59.989311 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fbc90c-63d6-45c9-a75c-9937c559e995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:59.990393 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbc90c-63d6-45c9-a75c-9937c559e995 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:59.990682 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbc90c-63d6-45c9-a75c-9937c559e995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:59.990955 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbc90c-63d6-45c9-a75c-9937c559e995 admin admin] PATH_INFO: `/v3/projects/be9ed042adc741ed89dbfb364d0aa13c/users/983d3640baf14f7ba60735e5c8452790/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:00.005658 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fbc90c-63d6-45c9-a75c-9937c559e995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be9ed042adc741ed89dbfb364d0aa13c, user_id=983d3640baf14f7ba60735e5c8452790, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:00.008370 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fbc90c-63d6-45c9-a75c-9937c559e995 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:00.024233 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1231/2461] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:49:59 2022] PUT /identity/v3/projects/be9ed042adc741ed89dbfb364d0aa13c/users/983d3640baf14f7ba60735e5c8452790/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:00.028191 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ee7cf-fbe8-4844-b547-0dd5390bd629 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:00.039024 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ee7cf-fbe8-4844-b547-0dd5390bd629 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:00.051530 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:00.097249 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c93311e-c7bb-41b4-91b8-509dc041a458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:00.097913 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93311e-c7bb-41b4-91b8-509dc041a458 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:00.098181 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93311e-c7bb-41b4-91b8-509dc041a458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:00.098489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93311e-c7bb-41b4-91b8-509dc041a458 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:00.099279 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c93311e-c7bb-41b4-91b8-509dc041a458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:00.100912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c93311e-c7bb-41b4-91b8-509dc041a458 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:00.110693 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1231/2462] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:00 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:00.115485 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fbc90c-63d6-45c9-a75c-9937c559e995 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:00.125928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fbc90c-63d6-45c9-a75c-9937c559e995 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:00.130775 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf75813-b39e-4275-99b3-0501738af507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:00.131641 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf75813-b39e-4275-99b3-0501738af507 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:00.131925 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf75813-b39e-4275-99b3-0501738af507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:00.132205 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf75813-b39e-4275-99b3-0501738af507 admin admin] PATH_INFO: `/v3/projects/be9ed042adc741ed89dbfb364d0aa13c/users/983d3640baf14f7ba60735e5c8452790/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:00.134211 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf75813-b39e-4275-99b3-0501738af507 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be9ed042adc741ed89dbfb364d0aa13c, user_id=983d3640baf14f7ba60735e5c8452790, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:00.136888 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf75813-b39e-4275-99b3-0501738af507 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:00.147313 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1232/2463] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:00 2022] PUT /identity/v3/projects/be9ed042adc741ed89dbfb364d0aa13c/users/983d3640baf14f7ba60735e5c8452790/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:00.157417 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24980b44-dac7-4d29-a8e2-8c99dfe5f9a5 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:00.157709 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24980b44-dac7-4d29-a8e2-8c99dfe5f9a5 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:00.158504 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24980b44-dac7-4d29-a8e2-8c99dfe5f9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:00.185675 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-24980b44-dac7-4d29-a8e2-8c99dfe5f9a5 None None] MFA Rules not processed for user `983d3640baf14f7ba60735e5c8452790`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:00.186877 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-24980b44-dac7-4d29-a8e2-8c99dfe5f9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:00.259320 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1232/2464] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:50:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:00.270687 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf75813-b39e-4275-99b3-0501738af507 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:00.272464 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-dbf75813-b39e-4275-99b3-0501738af507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:00.293523 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:00.334196 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf75813-b39e-4275-99b3-0501738af507 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:00.338019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a065d-9ced-4d87-bf15-ee90d05f0b32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:00.338690 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a065d-9ced-4d87-bf15-ee90d05f0b32 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:00.339098 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a065d-9ced-4d87-bf15-ee90d05f0b32 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:00.339288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a065d-9ced-4d87-bf15-ee90d05f0b32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:00.350023 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87a065d-9ced-4d87-bf15-ee90d05f0b32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:00.351538 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87a065d-9ced-4d87-bf15-ee90d05f0b32 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:00.361710 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1233/2465] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:00.887241 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24980b44-dac7-4d29-a8e2-8c99dfe5f9a5 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:00.888643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-24980b44-dac7-4d29-a8e2-8c99dfe5f9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:00.901258 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24980b44-dac7-4d29-a8e2-8c99dfe5f9a5 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:00.904797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c14313-18bf-4161-a236-68b6f14df8fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:00.905428 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c14313-18bf-4161-a236-68b6f14df8fd service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:00.905715 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c14313-18bf-4161-a236-68b6f14df8fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:00.905995 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c14313-18bf-4161-a236-68b6f14df8fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:00.964278 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c14313-18bf-4161-a236-68b6f14df8fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:00.965851 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c14313-18bf-4161-a236-68b6f14df8fd service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:00.976076 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1233/2466] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:04.965888 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a065d-9ced-4d87-bf15-ee90d05f0b32 service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:04.975750 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a065d-9ced-4d87-bf15-ee90d05f0b32 service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:04.979404 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1117ee11-180f-43e7-9296-6729d316d0d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:04.980035 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117ee11-180f-43e7-9296-6729d316d0d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:04.980315 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117ee11-180f-43e7-9296-6729d316d0d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:04.980588 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117ee11-180f-43e7-9296-6729d316d0d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:05.037807 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1117ee11-180f-43e7-9296-6729d316d0d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:05.039389 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1117ee11-180f-43e7-9296-6729d316d0d7 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:05.049959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1234/2467] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:07.163908 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c14313-18bf-4161-a236-68b6f14df8fd service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:07.205077 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:07.234939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c14313-18bf-4161-a236-68b6f14df8fd service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:07.239616 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea26394-abdd-47ee-a9fd-ac612c1dab15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:07.240487 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea26394-abdd-47ee-a9fd-ac612c1dab15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:07.240774 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea26394-abdd-47ee-a9fd-ac612c1dab15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:07.241053 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea26394-abdd-47ee-a9fd-ac612c1dab15 admin admin] PATH_INFO: `/v3/users/e8e1e57c0f68419dad301ab4f30ba7a3` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:07.242245 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea26394-abdd-47ee-a9fd-ac612c1dab15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8e1e57c0f68419dad301ab4f30ba7a3)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:07.244225 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea26394-abdd-47ee-a9fd-ac612c1dab15 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:07.303731 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-fea26394-abdd-47ee-a9fd-ac612c1dab15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8e1e57c0f68419dad301ab4f30ba7a3'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:07.304391 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-fea26394-abdd-47ee-a9fd-ac612c1dab15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8e1e57c0f68419dad301ab4f30ba7a3'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:07.317479 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-fea26394-abdd-47ee-a9fd-ac612c1dab15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8e1e57c0f68419dad301ab4f30ba7a3'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:07.326187 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-fea26394-abdd-47ee-a9fd-ac612c1dab15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8e1e57c0f68419dad301ab4f30ba7a3'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:07.333932 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-fea26394-abdd-47ee-a9fd-ac612c1dab15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8e1e57c0f68419dad301ab4f30ba7a3'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:07.340063 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1234/2468] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:50:07 2022] DELETE /identity/v3/users/e8e1e57c0f68419dad301ab4f30ba7a3 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:07.344867 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e9552-c21c-4842-a271-62eecd8adad3 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:07.346011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e9552-c21c-4842-a271-62eecd8adad3 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:07.346388 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e9552-c21c-4842-a271-62eecd8adad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:07.364576 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-085e9552-c21c-4842-a271-62eecd8adad3 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:07.365804 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-085e9552-c21c-4842-a271-62eecd8adad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:07.427863 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1235/2469] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:50:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:50:07.437558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea26394-abdd-47ee-a9fd-ac612c1dab15 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:07.439799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-fea26394-abdd-47ee-a9fd-ac612c1dab15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:07.498026 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea26394-abdd-47ee-a9fd-ac612c1dab15 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:07.502594 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606471c2-1458-4d30-bf2e-0f610c5774c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:07.503296 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606471c2-1458-4d30-bf2e-0f610c5774c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:07.503582 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606471c2-1458-4d30-bf2e-0f610c5774c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:07.503857 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606471c2-1458-4d30-bf2e-0f610c5774c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:07.513109 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606471c2-1458-4d30-bf2e-0f610c5774c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:07.514561 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606471c2-1458-4d30-bf2e-0f610c5774c4 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:07.525234 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1235/2470] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:07.818279 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085e9552-c21c-4842-a271-62eecd8adad3 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:07.820060 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-085e9552-c21c-4842-a271-62eecd8adad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:07.833693 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085e9552-c21c-4842-a271-62eecd8adad3 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:07.839119 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa798e25-1a7f-4148-a81e-ec69e61551f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:07.840006 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa798e25-1a7f-4148-a81e-ec69e61551f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:07.840302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa798e25-1a7f-4148-a81e-ec69e61551f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:07.840571 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa798e25-1a7f-4148-a81e-ec69e61551f3 admin admin] PATH_INFO: `/v3/projects/cef572b6559d4e3a98082cd4ef500b2c` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:07.841850 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa798e25-1a7f-4148-a81e-ec69e61551f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cef572b6559d4e3a98082cd4ef500b2c)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:07.843834 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa798e25-1a7f-4148-a81e-ec69e61551f3 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:07.898436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-aa798e25-1a7f-4148-a81e-ec69e61551f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cef572b6559d4e3a98082cd4ef500b2c', 'request_id': 'req-aa798e25-1a7f-4148-a81e-ec69e61551f3'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:07.901157 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:07.907503 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-aa798e25-1a7f-4148-a81e-ec69e61551f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cef572b6559d4e3a98082cd4ef500b2c', 'request_id': 'req-aa798e25-1a7f-4148-a81e-ec69e61551f3'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:07.917142 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-aa798e25-1a7f-4148-a81e-ec69e61551f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cef572b6559d4e3a98082cd4ef500b2c', 'request_id': 'req-aa798e25-1a7f-4148-a81e-ec69e61551f3'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:07.929393 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-aa798e25-1a7f-4148-a81e-ec69e61551f3 admin admin] The token cache is being invalidate because project cef572b6559d4e3a98082cd4ef500b2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:50:07.929776 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-aa798e25-1a7f-4148-a81e-ec69e61551f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:07.930475 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-aa798e25-1a7f-4148-a81e-ec69e61551f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:07.934231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1236/2471] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:50:07 2022] DELETE /identity/v3/projects/cef572b6559d4e3a98082cd4ef500b2c => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:07.938507 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606471c2-1458-4d30-bf2e-0f610c5774c4 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:07.948517 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606471c2-1458-4d30-bf2e-0f610c5774c4 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:08.015961 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:08.031183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6f0f1-1ab7-468d-b365-ee4c21b92a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:08.032082 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6f0f1-1ab7-468d-b365-ee4c21b92a32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:08.032372 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6f0f1-1ab7-468d-b365-ee4c21b92a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:08.032651 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6f0f1-1ab7-468d-b365-ee4c21b92a32 admin admin] PATH_INFO: `/v3/users/b03368f828b2424b90e7e72c84070576` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:08.034553 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce6f0f1-1ab7-468d-b365-ee4c21b92a32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b03368f828b2424b90e7e72c84070576)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:08.036843 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce6f0f1-1ab7-468d-b365-ee4c21b92a32 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:08.086972 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-2ce6f0f1-1ab7-468d-b365-ee4c21b92a32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b03368f828b2424b90e7e72c84070576'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:08.087778 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-2ce6f0f1-1ab7-468d-b365-ee4c21b92a32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b03368f828b2424b90e7e72c84070576'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:08.105683 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-2ce6f0f1-1ab7-468d-b365-ee4c21b92a32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b03368f828b2424b90e7e72c84070576'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:08.114758 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-2ce6f0f1-1ab7-468d-b365-ee4c21b92a32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b03368f828b2424b90e7e72c84070576'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:08.122411 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-2ce6f0f1-1ab7-468d-b365-ee4c21b92a32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b03368f828b2424b90e7e72c84070576'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:08.132395 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1236/2472] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:50:07 2022] DELETE /identity/v3/users/b03368f828b2424b90e7e72c84070576 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:08.446330 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa798e25-1a7f-4148-a81e-ec69e61551f3 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:08.449178 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-aa798e25-1a7f-4148-a81e-ec69e61551f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:08.469446 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa798e25-1a7f-4148-a81e-ec69e61551f3 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:08.548512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e977be4-0b1e-4dca-9980-8c0b4a5e0e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:08.549694 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e977be4-0b1e-4dca-9980-8c0b4a5e0e46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:08.550054 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e977be4-0b1e-4dca-9980-8c0b4a5e0e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:08.550406 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e977be4-0b1e-4dca-9980-8c0b4a5e0e46 admin admin] PATH_INFO: `/v3/projects/0238208299734f04826c1590b12e1c0f` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:08.551817 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e977be4-0b1e-4dca-9980-8c0b4a5e0e46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0238208299734f04826c1590b12e1c0f)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:08.553871 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e977be4-0b1e-4dca-9980-8c0b4a5e0e46 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:08.614484 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-8e977be4-0b1e-4dca-9980-8c0b4a5e0e46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0238208299734f04826c1590b12e1c0f', 'request_id': 'req-8e977be4-0b1e-4dca-9980-8c0b4a5e0e46'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:08.624764 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-8e977be4-0b1e-4dca-9980-8c0b4a5e0e46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0238208299734f04826c1590b12e1c0f', 'request_id': 'req-8e977be4-0b1e-4dca-9980-8c0b4a5e0e46'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:08.633794 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-8e977be4-0b1e-4dca-9980-8c0b4a5e0e46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0238208299734f04826c1590b12e1c0f', 'request_id': 'req-8e977be4-0b1e-4dca-9980-8c0b4a5e0e46'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:08.647506 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-8e977be4-0b1e-4dca-9980-8c0b4a5e0e46 admin admin] The token cache is being invalidate because project 0238208299734f04826c1590b12e1c0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:50:08.647916 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-8e977be4-0b1e-4dca-9980-8c0b4a5e0e46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:08.648671 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-8e977be4-0b1e-4dca-9980-8c0b4a5e0e46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:08.650876 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1237/2473] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:50:08 2022] DELETE /identity/v3/projects/0238208299734f04826c1590b12e1c0f => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:08.654750 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6f0f1-1ab7-468d-b365-ee4c21b92a32 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:08.656362 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-2ce6f0f1-1ab7-468d-b365-ee4c21b92a32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:08.673950 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6f0f1-1ab7-468d-b365-ee4c21b92a32 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:08.750629 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a9fba-8d9c-4623-8549-0d19473eecff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:08.752237 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a9fba-8d9c-4623-8549-0d19473eecff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:08.752533 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a9fba-8d9c-4623-8549-0d19473eecff admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:08.752929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a9fba-8d9c-4623-8549-0d19473eecff admin admin] PATH_INFO: `/v3/users/983d3640baf14f7ba60735e5c8452790` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:08.754241 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a9fba-8d9c-4623-8549-0d19473eecff admin admin] RBAC: Authorizing `identity:delete_user(user_id=983d3640baf14f7ba60735e5c8452790)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:08.756241 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a9fba-8d9c-4623-8549-0d19473eecff admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:08.816824 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-f86a9fba-8d9c-4623-8549-0d19473eecff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '983d3640baf14f7ba60735e5c8452790'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:08.817585 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-f86a9fba-8d9c-4623-8549-0d19473eecff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '983d3640baf14f7ba60735e5c8452790'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:08.834720 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-f86a9fba-8d9c-4623-8549-0d19473eecff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '983d3640baf14f7ba60735e5c8452790'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:08.847043 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-f86a9fba-8d9c-4623-8549-0d19473eecff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '983d3640baf14f7ba60735e5c8452790'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:08.854391 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:08.856984 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-f86a9fba-8d9c-4623-8549-0d19473eecff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '983d3640baf14f7ba60735e5c8452790'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:08.864908 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1237/2474] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:50:08 2022] DELETE /identity/v3/users/983d3640baf14f7ba60735e5c8452790 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:50:09.244831 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e977be4-0b1e-4dca-9980-8c0b4a5e0e46 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:09.246342 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-8e977be4-0b1e-4dca-9980-8c0b4a5e0e46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:09.265303 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e977be4-0b1e-4dca-9980-8c0b4a5e0e46 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:09.323754 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49e13f-64f6-4710-a723-001127138722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:09.324537 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49e13f-64f6-4710-a723-001127138722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:09.324815 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49e13f-64f6-4710-a723-001127138722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:09.325092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49e13f-64f6-4710-a723-001127138722 admin admin] PATH_INFO: `/v3/projects/be9ed042adc741ed89dbfb364d0aa13c` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:09.326322 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee49e13f-64f6-4710-a723-001127138722 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be9ed042adc741ed89dbfb364d0aa13c)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:09.328485 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee49e13f-64f6-4710-a723-001127138722 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:09.339679 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:09.375943 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ee49e13f-64f6-4710-a723-001127138722 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be9ed042adc741ed89dbfb364d0aa13c', 'request_id': 'req-ee49e13f-64f6-4710-a723-001127138722'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:09.383225 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ee49e13f-64f6-4710-a723-001127138722 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be9ed042adc741ed89dbfb364d0aa13c', 'request_id': 'req-ee49e13f-64f6-4710-a723-001127138722'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:09.389680 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ee49e13f-64f6-4710-a723-001127138722 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be9ed042adc741ed89dbfb364d0aa13c', 'request_id': 'req-ee49e13f-64f6-4710-a723-001127138722'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:09.400205 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ee49e13f-64f6-4710-a723-001127138722 admin admin] The token cache is being invalidate because project be9ed042adc741ed89dbfb364d0aa13c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:50:09.400487 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ee49e13f-64f6-4710-a723-001127138722 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:09.401123 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ee49e13f-64f6-4710-a723-001127138722 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:09.402675 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1238/2475] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:50:09 2022] DELETE /identity/v3/projects/be9ed042adc741ed89dbfb364d0aa13c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:09.410220 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052439a-091a-454f-8597-77116d3160e0 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:09.410513 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052439a-091a-454f-8597-77116d3160e0 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:09.411076 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052439a-091a-454f-8597-77116d3160e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:09.429900 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-e052439a-091a-454f-8597-77116d3160e0 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:09.431155 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e052439a-091a-454f-8597-77116d3160e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:09.490509 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1238/2476] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:50:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:09.499133 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4726bc-b40b-401c-b7d8-910f53630059 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:09.499631 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4726bc-b40b-401c-b7d8-910f53630059 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:09.499741 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4726bc-b40b-401c-b7d8-910f53630059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:09.517606 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-3a4726bc-b40b-401c-b7d8-910f53630059 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:09.518798 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-3a4726bc-b40b-401c-b7d8-910f53630059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:09.526922 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1239/2477] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:50:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:50:09.531188 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e052439a-091a-454f-8597-77116d3160e0 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:09.541790 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e052439a-091a-454f-8597-77116d3160e0 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:09.546503 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4324a-a761-4534-85f5-4da03c2658b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:09.547190 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4324a-a761-4534-85f5-4da03c2658b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:09.547475 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4324a-a761-4534-85f5-4da03c2658b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:09.547752 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4324a-a761-4534-85f5-4da03c2658b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:09.549256 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec4324a-a761-4534-85f5-4da03c2658b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:50:09.549564 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec4324a-a761-4534-85f5-4da03c2658b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:09.551183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec4324a-a761-4534-85f5-4da03c2658b2 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:09.562660 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1239/2478] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:50:09.566837 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4726bc-b40b-401c-b7d8-910f53630059 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:09.575964 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4726bc-b40b-401c-b7d8-910f53630059 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:09.580650 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e2653-9f66-4b8c-8005-74da9f277b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:09.581286 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e2653-9f66-4b8c-8005-74da9f277b7b admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:09.581566 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e2653-9f66-4b8c-8005-74da9f277b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:09.581840 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e2653-9f66-4b8c-8005-74da9f277b7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:09.582529 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340e2653-9f66-4b8c-8005-74da9f277b7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:09.584474 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340e2653-9f66-4b8c-8005-74da9f277b7b admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:09.610499 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1240/2479] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:50:09 2022] POST /identity/v3/projects => generated 409 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:09.615231 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4324a-a761-4534-85f5-4da03c2658b2 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:09.625068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4324a-a761-4534-85f5-4da03c2658b2 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:09.681179 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d122a21a-72c9-4cda-ad2d-d764c9e22ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:09.681841 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122a21a-72c9-4cda-ad2d-d764c9e22ac8 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:09.682109 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122a21a-72c9-4cda-ad2d-d764c9e22ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:09.682382 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122a21a-72c9-4cda-ad2d-d764c9e22ac8 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:09.683099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d122a21a-72c9-4cda-ad2d-d764c9e22ac8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:09.685014 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d122a21a-72c9-4cda-ad2d-d764c9e22ac8 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:09.704389 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1240/2480] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:50:09 2022] POST /identity/v3/users => generated 305 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:09.709126 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e2653-9f66-4b8c-8005-74da9f277b7b admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:09.719476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e2653-9f66-4b8c-8005-74da9f277b7b admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:09.725056 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a59fcc2-201d-491b-ac6e-9232ac325f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:09.725715 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59fcc2-201d-491b-ac6e-9232ac325f17 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:09.725995 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59fcc2-201d-491b-ac6e-9232ac325f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:09.726270 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59fcc2-201d-491b-ac6e-9232ac325f17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:09.727017 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a59fcc2-201d-491b-ac6e-9232ac325f17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:09.728720 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a59fcc2-201d-491b-ac6e-9232ac325f17 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:09.739268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1241/2481] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:09.743444 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d122a21a-72c9-4cda-ad2d-d764c9e22ac8 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:09.753543 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d122a21a-72c9-4cda-ad2d-d764c9e22ac8 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:09.758123 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed349ec-caf5-4c48-acb9-f24ea8968295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:09.758770 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed349ec-caf5-4c48-acb9-f24ea8968295 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:09.759080 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed349ec-caf5-4c48-acb9-f24ea8968295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:09.759359 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed349ec-caf5-4c48-acb9-f24ea8968295 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:09.760092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed349ec-caf5-4c48-acb9-f24ea8968295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:09.761659 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed349ec-caf5-4c48-acb9-f24ea8968295 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:09.772688 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1241/2482] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:09.776859 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a59fcc2-201d-491b-ac6e-9232ac325f17 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:09.787926 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a59fcc2-201d-491b-ac6e-9232ac325f17 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:09.792870 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d428fbe8-8268-4b0b-93dd-6ded9cb4d103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:09.793767 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428fbe8-8268-4b0b-93dd-6ded9cb4d103 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:09.794055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428fbe8-8268-4b0b-93dd-6ded9cb4d103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:09.794332 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428fbe8-8268-4b0b-93dd-6ded9cb4d103 admin admin] PATH_INFO: `/v3/projects/945b724d2475462eba3f7ff032f0eefe/users/365fcd3d17414a59a106ef88d0278367/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:09.809192 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d428fbe8-8268-4b0b-93dd-6ded9cb4d103 admin admin] RBAC: Authorizing `identity:create_grant(project_id=945b724d2475462eba3f7ff032f0eefe, user_id=365fcd3d17414a59a106ef88d0278367, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:09.811899 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d428fbe8-8268-4b0b-93dd-6ded9cb4d103 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:09.825415 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1242/2483] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:09 2022] PUT /identity/v3/projects/945b724d2475462eba3f7ff032f0eefe/users/365fcd3d17414a59a106ef88d0278367/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:09.838394 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82b21c-35cf-4317-a229-7c98cb6b1ccd None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:09.838650 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82b21c-35cf-4317-a229-7c98cb6b1ccd None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:09.838945 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82b21c-35cf-4317-a229-7c98cb6b1ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:09.868155 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-ea82b21c-35cf-4317-a229-7c98cb6b1ccd None None] MFA Rules not processed for user `365fcd3d17414a59a106ef88d0278367`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:09.869395 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ea82b21c-35cf-4317-a229-7c98cb6b1ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:09.918551 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:09.935924 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1242/2484] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:50:09 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:09.940519 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d428fbe8-8268-4b0b-93dd-6ded9cb4d103 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:10.000383 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d428fbe8-8268-4b0b-93dd-6ded9cb4d103 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:10.005179 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be6f4f6-db6c-4872-831e-0808585d9f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:10.005835 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6f4f6-db6c-4872-831e-0808585d9f37 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:10.006116 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6f4f6-db6c-4872-831e-0808585d9f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:10.006390 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6f4f6-db6c-4872-831e-0808585d9f37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:10.007109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be6f4f6-db6c-4872-831e-0808585d9f37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:10.009025 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be6f4f6-db6c-4872-831e-0808585d9f37 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:10.039016 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1243/2485] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:50:09 2022] POST /identity/v3/projects => generated 409 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:10.042888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82b21c-35cf-4317-a229-7c98cb6b1ccd None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:10.053410 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82b21c-35cf-4317-a229-7c98cb6b1ccd None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:10.110123 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4e7bd-563e-4698-ba8c-c155023fbe0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:10.110701 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4e7bd-563e-4698-ba8c-c155023fbe0b admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:10.110984 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4e7bd-563e-4698-ba8c-c155023fbe0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:10.111309 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4e7bd-563e-4698-ba8c-c155023fbe0b admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:10.112290 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4e7bd-563e-4698-ba8c-c155023fbe0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:10.114367 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4e7bd-563e-4698-ba8c-c155023fbe0b admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:10.135292 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1243/2486] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:50:10 2022] POST /identity/v3/users => generated 311 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:10.139173 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be6f4f6-db6c-4872-831e-0808585d9f37 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:10.150090 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be6f4f6-db6c-4872-831e-0808585d9f37 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:10.154747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b4655-36eb-44c3-baf7-a2f2005cadc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:10.155445 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b4655-36eb-44c3-baf7-a2f2005cadc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:10.155728 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b4655-36eb-44c3-baf7-a2f2005cadc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:10.156006 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b4655-36eb-44c3-baf7-a2f2005cadc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:10.156764 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b4655-36eb-44c3-baf7-a2f2005cadc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:10.158362 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b4655-36eb-44c3-baf7-a2f2005cadc2 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:10.169222 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1244/2487] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:10.173537 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4e7bd-563e-4698-ba8c-c155023fbe0b admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:10.183800 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4e7bd-563e-4698-ba8c-c155023fbe0b admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:10.188430 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e1c26c-6149-4b79-b3b9-c7d6eb2d5ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:10.189819 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1c26c-6149-4b79-b3b9-c7d6eb2d5ffc admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:10.190103 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1c26c-6149-4b79-b3b9-c7d6eb2d5ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:10.190378 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1c26c-6149-4b79-b3b9-c7d6eb2d5ffc admin admin] PATH_INFO: `/v3/projects/333e171764d9491099429b7dcd70a967/users/3a3e0273454b44219040f1a7df1c134e/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:10.206015 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e1c26c-6149-4b79-b3b9-c7d6eb2d5ffc admin admin] RBAC: Authorizing `identity:create_grant(project_id=333e171764d9491099429b7dcd70a967, user_id=3a3e0273454b44219040f1a7df1c134e, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:10.208850 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e1c26c-6149-4b79-b3b9-c7d6eb2d5ffc admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:10.225950 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1244/2488] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:10 2022] PUT /identity/v3/projects/333e171764d9491099429b7dcd70a967/users/3a3e0273454b44219040f1a7df1c134e/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:50:10.230199 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b4655-36eb-44c3-baf7-a2f2005cadc2 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:10.240127 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b4655-36eb-44c3-baf7-a2f2005cadc2 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:10.247751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:10.303394 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2390237-9db4-415d-84c6-858221b135de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:10.304063 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2390237-9db4-415d-84c6-858221b135de admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:10.304345 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2390237-9db4-415d-84c6-858221b135de admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:10.304618 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2390237-9db4-415d-84c6-858221b135de admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:10.305336 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2390237-9db4-415d-84c6-858221b135de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:10.306943 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2390237-9db4-415d-84c6-858221b135de admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:10.317250 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1245/2489] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:10.321530 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e1c26c-6149-4b79-b3b9-c7d6eb2d5ffc admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:10.332532 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e1c26c-6149-4b79-b3b9-c7d6eb2d5ffc admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:10.337194 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393e661-a115-43c2-89c9-a601b4d8212e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:10.338057 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393e661-a115-43c2-89c9-a601b4d8212e admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:10.338337 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393e661-a115-43c2-89c9-a601b4d8212e admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:10.338607 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393e661-a115-43c2-89c9-a601b4d8212e admin admin] PATH_INFO: `/v3/projects/333e171764d9491099429b7dcd70a967/users/3a3e0273454b44219040f1a7df1c134e/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:10.340574 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3393e661-a115-43c2-89c9-a601b4d8212e admin admin] RBAC: Authorizing `identity:create_grant(project_id=333e171764d9491099429b7dcd70a967, user_id=3a3e0273454b44219040f1a7df1c134e, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:10.343200 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3393e661-a115-43c2-89c9-a601b4d8212e admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:10.353905 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1245/2490] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:10 2022] PUT /identity/v3/projects/333e171764d9491099429b7dcd70a967/users/3a3e0273454b44219040f1a7df1c134e/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:10.364317 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6189b1-1dd9-416a-ba5a-9f8039b893dd None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:10.364548 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6189b1-1dd9-416a-ba5a-9f8039b893dd None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:10.364832 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6189b1-1dd9-416a-ba5a-9f8039b893dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:10.391068 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-af6189b1-1dd9-416a-ba5a-9f8039b893dd None None] MFA Rules not processed for user `3a3e0273454b44219040f1a7df1c134e`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:10.392279 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-af6189b1-1dd9-416a-ba5a-9f8039b893dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:10.460040 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1246/2491] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:50:10 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:10.470884 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393e661-a115-43c2-89c9-a601b4d8212e admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:10.472636 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-3393e661-a115-43c2-89c9-a601b4d8212e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:10.535466 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393e661-a115-43c2-89c9-a601b4d8212e admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:10.539351 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3044b619-a950-41a4-9d9d-de7be8a2de18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:10.539985 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044b619-a950-41a4-9d9d-de7be8a2de18 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:10.540266 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044b619-a950-41a4-9d9d-de7be8a2de18 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:10.540541 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044b619-a950-41a4-9d9d-de7be8a2de18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:10.550232 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3044b619-a950-41a4-9d9d-de7be8a2de18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:10.551724 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3044b619-a950-41a4-9d9d-de7be8a2de18 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:10.561327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1246/2492] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:10 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:10.714785 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6189b1-1dd9-416a-ba5a-9f8039b893dd None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:10.724715 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6189b1-1dd9-416a-ba5a-9f8039b893dd None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:10.728534 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d809097f-0d89-48fc-99a2-34a6bcc5e5f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:10.729200 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d809097f-0d89-48fc-99a2-34a6bcc5e5f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:10.729467 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d809097f-0d89-48fc-99a2-34a6bcc5e5f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:10.729744 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d809097f-0d89-48fc-99a2-34a6bcc5e5f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:10.788715 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d809097f-0d89-48fc-99a2-34a6bcc5e5f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:10.790163 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d809097f-0d89-48fc-99a2-34a6bcc5e5f4 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:10.799851 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1247/2493] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:10 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:20.124445 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3044b619-a950-41a4-9d9d-de7be8a2de18 service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:20.173299 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:20.192359 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3044b619-a950-41a4-9d9d-de7be8a2de18 service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:20.196929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552bfb5f-b4c1-4b35-bc55-efd6e5ed2fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:20.197808 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552bfb5f-b4c1-4b35-bc55-efd6e5ed2fb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:20.198094 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552bfb5f-b4c1-4b35-bc55-efd6e5ed2fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:20.198374 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552bfb5f-b4c1-4b35-bc55-efd6e5ed2fb4 admin admin] PATH_INFO: `/v3/users/365fcd3d17414a59a106ef88d0278367` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:20.199614 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552bfb5f-b4c1-4b35-bc55-efd6e5ed2fb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=365fcd3d17414a59a106ef88d0278367)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:20.201541 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552bfb5f-b4c1-4b35-bc55-efd6e5ed2fb4 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:20.252694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-552bfb5f-b4c1-4b35-bc55-efd6e5ed2fb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '365fcd3d17414a59a106ef88d0278367'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:20.253276 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-552bfb5f-b4c1-4b35-bc55-efd6e5ed2fb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '365fcd3d17414a59a106ef88d0278367'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:20.266066 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-552bfb5f-b4c1-4b35-bc55-efd6e5ed2fb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '365fcd3d17414a59a106ef88d0278367'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:20.274471 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-552bfb5f-b4c1-4b35-bc55-efd6e5ed2fb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '365fcd3d17414a59a106ef88d0278367'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:20.281564 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-552bfb5f-b4c1-4b35-bc55-efd6e5ed2fb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '365fcd3d17414a59a106ef88d0278367'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:20.288376 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1247/2494] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:50:20 2022] DELETE /identity/v3/users/365fcd3d17414a59a106ef88d0278367 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:20.293079 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4243d8-6b4b-4820-b2df-86eaf6b11261 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:20.293562 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4243d8-6b4b-4820-b2df-86eaf6b11261 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:20.293673 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4243d8-6b4b-4820-b2df-86eaf6b11261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:20.310956 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-5b4243d8-6b4b-4820-b2df-86eaf6b11261 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:20.312203 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5b4243d8-6b4b-4820-b2df-86eaf6b11261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:20.339742 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:20.372732 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1248/2495] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:50:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:20.383790 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552bfb5f-b4c1-4b35-bc55-efd6e5ed2fb4 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:20.385212 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-552bfb5f-b4c1-4b35-bc55-efd6e5ed2fb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:20.443264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552bfb5f-b4c1-4b35-bc55-efd6e5ed2fb4 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:20.447040 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac4881-171f-4039-953b-c90ca9b9361f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:20.447707 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac4881-171f-4039-953b-c90ca9b9361f service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:20.448022 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac4881-171f-4039-953b-c90ca9b9361f service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:20.448334 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac4881-171f-4039-953b-c90ca9b9361f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:20.458449 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac4881-171f-4039-953b-c90ca9b9361f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:20.459929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac4881-171f-4039-953b-c90ca9b9361f service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:20.469501 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1248/2496] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:20.741187 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4243d8-6b4b-4820-b2df-86eaf6b11261 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:20.742630 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5b4243d8-6b4b-4820-b2df-86eaf6b11261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:20.754664 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4243d8-6b4b-4820-b2df-86eaf6b11261 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:20.759592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31918e-4e8f-4a28-91bf-e7d595fc225a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:20.760665 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31918e-4e8f-4a28-91bf-e7d595fc225a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:20.760943 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31918e-4e8f-4a28-91bf-e7d595fc225a admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:20.761215 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31918e-4e8f-4a28-91bf-e7d595fc225a admin admin] PATH_INFO: `/v3/projects/945b724d2475462eba3f7ff032f0eefe` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:20.762427 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31918e-4e8f-4a28-91bf-e7d595fc225a admin admin] RBAC: Authorizing `identity:delete_project(project_id=945b724d2475462eba3f7ff032f0eefe)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:20.764702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31918e-4e8f-4a28-91bf-e7d595fc225a admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:20.817490 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ce31918e-4e8f-4a28-91bf-e7d595fc225a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '945b724d2475462eba3f7ff032f0eefe', 'request_id': 'req-ce31918e-4e8f-4a28-91bf-e7d595fc225a'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:20.825702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ce31918e-4e8f-4a28-91bf-e7d595fc225a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '945b724d2475462eba3f7ff032f0eefe', 'request_id': 'req-ce31918e-4e8f-4a28-91bf-e7d595fc225a'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:20.832252 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ce31918e-4e8f-4a28-91bf-e7d595fc225a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '945b724d2475462eba3f7ff032f0eefe', 'request_id': 'req-ce31918e-4e8f-4a28-91bf-e7d595fc225a'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:20.842763 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ce31918e-4e8f-4a28-91bf-e7d595fc225a admin admin] The token cache is being invalidate because project 945b724d2475462eba3f7ff032f0eefe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:50:20.843079 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ce31918e-4e8f-4a28-91bf-e7d595fc225a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:20.843724 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ce31918e-4e8f-4a28-91bf-e7d595fc225a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:20.845272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1249/2497] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:50:20 2022] DELETE /identity/v3/projects/945b724d2475462eba3f7ff032f0eefe => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:20.851522 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac4881-171f-4039-953b-c90ca9b9361f service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:20.861978 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac4881-171f-4039-953b-c90ca9b9361f service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:20.918534 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f8aaf-f4f2-4319-8fe0-cab8f4e09704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:20.919517 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f8aaf-f4f2-4319-8fe0-cab8f4e09704 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:20.919803 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f8aaf-f4f2-4319-8fe0-cab8f4e09704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:20.920084 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f8aaf-f4f2-4319-8fe0-cab8f4e09704 admin admin] PATH_INFO: `/v3/users/3a3e0273454b44219040f1a7df1c134e` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:20.921278 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f8aaf-f4f2-4319-8fe0-cab8f4e09704 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a3e0273454b44219040f1a7df1c134e)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:20.923233 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f8aaf-f4f2-4319-8fe0-cab8f4e09704 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:20.933875 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:20.971389 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-7e7f8aaf-f4f2-4319-8fe0-cab8f4e09704 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a3e0273454b44219040f1a7df1c134e'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:20.972050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-7e7f8aaf-f4f2-4319-8fe0-cab8f4e09704 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a3e0273454b44219040f1a7df1c134e'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:20.984100 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-7e7f8aaf-f4f2-4319-8fe0-cab8f4e09704 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a3e0273454b44219040f1a7df1c134e'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:20.991575 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-7e7f8aaf-f4f2-4319-8fe0-cab8f4e09704 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a3e0273454b44219040f1a7df1c134e'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:20.998545 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-7e7f8aaf-f4f2-4319-8fe0-cab8f4e09704 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a3e0273454b44219040f1a7df1c134e'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:21.004571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1249/2498] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:50:20 2022] DELETE /identity/v3/users/3a3e0273454b44219040f1a7df1c134e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:21.275054 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31918e-4e8f-4a28-91bf-e7d595fc225a admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:21.276756 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ce31918e-4e8f-4a28-91bf-e7d595fc225a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:21.291272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31918e-4e8f-4a28-91bf-e7d595fc225a admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:21.349180 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1723c5c-9383-45c1-8a8a-4448e67930a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:21.350038 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1723c5c-9383-45c1-8a8a-4448e67930a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:21.350319 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1723c5c-9383-45c1-8a8a-4448e67930a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:21.350590 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1723c5c-9383-45c1-8a8a-4448e67930a9 admin admin] PATH_INFO: `/v3/projects/333e171764d9491099429b7dcd70a967` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:21.351904 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1723c5c-9383-45c1-8a8a-4448e67930a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=333e171764d9491099429b7dcd70a967)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:21.353747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1723c5c-9383-45c1-8a8a-4448e67930a9 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:21.399180 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b1723c5c-9383-45c1-8a8a-4448e67930a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '333e171764d9491099429b7dcd70a967', 'request_id': 'req-b1723c5c-9383-45c1-8a8a-4448e67930a9'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:21.406677 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b1723c5c-9383-45c1-8a8a-4448e67930a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '333e171764d9491099429b7dcd70a967', 'request_id': 'req-b1723c5c-9383-45c1-8a8a-4448e67930a9'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:21.407998 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:21.415760 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b1723c5c-9383-45c1-8a8a-4448e67930a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '333e171764d9491099429b7dcd70a967', 'request_id': 'req-b1723c5c-9383-45c1-8a8a-4448e67930a9'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:21.427973 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b1723c5c-9383-45c1-8a8a-4448e67930a9 admin admin] The token cache is being invalidate because project 333e171764d9491099429b7dcd70a967 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:50:21.428260 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b1723c5c-9383-45c1-8a8a-4448e67930a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:21.428939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b1723c5c-9383-45c1-8a8a-4448e67930a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:21.430486 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1250/2499] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:50:21 2022] DELETE /identity/v3/projects/333e171764d9491099429b7dcd70a967 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:21.435559 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c3f7d-38d4-46a6-a592-d6720267fd35 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:21.436011 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c3f7d-38d4-46a6-a592-d6720267fd35 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:21.436156 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c3f7d-38d4-46a6-a592-d6720267fd35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:21.453994 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-e44c3f7d-38d4-46a6-a592-d6720267fd35 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:21.455892 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e44c3f7d-38d4-46a6-a592-d6720267fd35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:21.516889 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1250/2500] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:50:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:21.526321 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535855a-89cb-46c1-b790-de061b94db2a None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:21.526649 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535855a-89cb-46c1-b790-de061b94db2a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:21.526930 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535855a-89cb-46c1-b790-de061b94db2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:21.545501 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-3535855a-89cb-46c1-b790-de061b94db2a None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:21.546700 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-3535855a-89cb-46c1-b790-de061b94db2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:21.554682 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1251/2501] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:50:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:21.559062 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c3f7d-38d4-46a6-a592-d6720267fd35 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:21.569289 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c3f7d-38d4-46a6-a592-d6720267fd35 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:21.573774 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ab881-7bda-4c36-8644-1bbb96f539d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:21.574411 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ab881-7bda-4c36-8644-1bbb96f539d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:21.574692 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ab881-7bda-4c36-8644-1bbb96f539d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:21.574968 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ab881-7bda-4c36-8644-1bbb96f539d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:21.575713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8ab881-7bda-4c36-8644-1bbb96f539d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:50:21.576020 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8ab881-7bda-4c36-8644-1bbb96f539d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:21.577615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8ab881-7bda-4c36-8644-1bbb96f539d1 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:21.590050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1251/2502] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:50:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:50:21.595459 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535855a-89cb-46c1-b790-de061b94db2a None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:21.604845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535855a-89cb-46c1-b790-de061b94db2a None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:21.609436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db6465-1942-46cc-a4a9-9bbfcd3b6c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:21.610080 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db6465-1942-46cc-a4a9-9bbfcd3b6c48 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:21.610361 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db6465-1942-46cc-a4a9-9bbfcd3b6c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:21.610636 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db6465-1942-46cc-a4a9-9bbfcd3b6c48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:21.611373 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db6465-1942-46cc-a4a9-9bbfcd3b6c48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:21.613306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db6465-1942-46cc-a4a9-9bbfcd3b6c48 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:21.639249 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1252/2503] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:50:21 2022] POST /identity/v3/projects => generated 365 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:21.644024 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ab881-7bda-4c36-8644-1bbb96f539d1 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:21.654330 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ab881-7bda-4c36-8644-1bbb96f539d1 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:21.712260 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b043e-3b6f-4fe6-992b-5757a6e56ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:21.712908 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b043e-3b6f-4fe6-992b-5757a6e56ccc admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:21.713192 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b043e-3b6f-4fe6-992b-5757a6e56ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:21.713535 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b043e-3b6f-4fe6-992b-5757a6e56ccc admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:21.714376 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07b043e-3b6f-4fe6-992b-5757a6e56ccc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:21.716384 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07b043e-3b6f-4fe6-992b-5757a6e56ccc admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:21.737908 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1252/2504] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:50:21 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:21.742225 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db6465-1942-46cc-a4a9-9bbfcd3b6c48 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:21.752973 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db6465-1942-46cc-a4a9-9bbfcd3b6c48 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:21.757631 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386eb911-3243-4580-a11e-9fc424f85d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:21.758281 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386eb911-3243-4580-a11e-9fc424f85d03 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:21.758556 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386eb911-3243-4580-a11e-9fc424f85d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:21.758834 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386eb911-3243-4580-a11e-9fc424f85d03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:21.759593 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386eb911-3243-4580-a11e-9fc424f85d03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:21.761190 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386eb911-3243-4580-a11e-9fc424f85d03 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:21.771847 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1253/2505] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:21.775969 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b043e-3b6f-4fe6-992b-5757a6e56ccc admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:21.782305 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:21.787098 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b043e-3b6f-4fe6-992b-5757a6e56ccc admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:21.794086 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe756ee4-fa79-4aa0-ac52-7f98f979cb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:21.795024 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe756ee4-fa79-4aa0-ac52-7f98f979cb2f admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:21.795378 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe756ee4-fa79-4aa0-ac52-7f98f979cb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:21.795838 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe756ee4-fa79-4aa0-ac52-7f98f979cb2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:21.796704 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe756ee4-fa79-4aa0-ac52-7f98f979cb2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:21.799031 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe756ee4-fa79-4aa0-ac52-7f98f979cb2f admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:21.809304 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1253/2506] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:50:21.813533 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386eb911-3243-4580-a11e-9fc424f85d03 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:21.824268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386eb911-3243-4580-a11e-9fc424f85d03 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:21.828978 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7aea3-54b6-48ce-b325-5ee2c28f7131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:21.829975 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7aea3-54b6-48ce-b325-5ee2c28f7131 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:21.830259 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7aea3-54b6-48ce-b325-5ee2c28f7131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:21.830535 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7aea3-54b6-48ce-b325-5ee2c28f7131 admin admin] PATH_INFO: `/v3/projects/ad19c4dcbcf24dc9bbc736cea10b18ac/users/30f26df76d2843768ee8456c4f2940cd/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:21.845231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be7aea3-54b6-48ce-b325-5ee2c28f7131 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad19c4dcbcf24dc9bbc736cea10b18ac, user_id=30f26df76d2843768ee8456c4f2940cd, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:21.847944 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be7aea3-54b6-48ce-b325-5ee2c28f7131 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:21.862197 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1254/2507] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:21 2022] PUT /identity/v3/projects/ad19c4dcbcf24dc9bbc736cea10b18ac/users/30f26df76d2843768ee8456c4f2940cd/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:50:21.872830 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab52ab-162a-4475-af17-dd315257aaa4 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:21.873126 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab52ab-162a-4475-af17-dd315257aaa4 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:21.873437 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab52ab-162a-4475-af17-dd315257aaa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:21.902758 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-45ab52ab-162a-4475-af17-dd315257aaa4 None None] MFA Rules not processed for user `30f26df76d2843768ee8456c4f2940cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:21.904009 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-45ab52ab-162a-4475-af17-dd315257aaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:21.970326 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1254/2508] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:50:21 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:21.974963 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7aea3-54b6-48ce-b325-5ee2c28f7131 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:22.036969 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7aea3-54b6-48ce-b325-5ee2c28f7131 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:22.041735 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e3da4-cfe9-45d8-8639-2f81295af6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:22.042727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e3da4-cfe9-45d8-8639-2f81295af6c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:22.042727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e3da4-cfe9-45d8-8639-2f81295af6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:22.042959 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e3da4-cfe9-45d8-8639-2f81295af6c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:22.043695 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1e3da4-cfe9-45d8-8639-2f81295af6c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:22.045687 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1e3da4-cfe9-45d8-8639-2f81295af6c3 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:22.073400 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1255/2509] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:50:21 2022] POST /identity/v3/projects => generated 367 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:22.077585 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab52ab-162a-4475-af17-dd315257aaa4 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:22.088188 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab52ab-162a-4475-af17-dd315257aaa4 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:22.144215 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65d5b3-3a8a-4906-9384-cc9bc4636875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:22.144996 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65d5b3-3a8a-4906-9384-cc9bc4636875 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:22.145291 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65d5b3-3a8a-4906-9384-cc9bc4636875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:22.145570 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65d5b3-3a8a-4906-9384-cc9bc4636875 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:22.146274 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65d5b3-3a8a-4906-9384-cc9bc4636875 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:22.148254 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65d5b3-3a8a-4906-9384-cc9bc4636875 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:22.170016 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1255/2510] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:50:22 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:22.174851 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e3da4-cfe9-45d8-8639-2f81295af6c3 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:22.186193 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e3da4-cfe9-45d8-8639-2f81295af6c3 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:22.191129 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a89c0-46d4-4b88-a808-ed5228f2a21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:22.191793 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a89c0-46d4-4b88-a808-ed5228f2a21c admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:22.192077 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a89c0-46d4-4b88-a808-ed5228f2a21c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:22.192352 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a89c0-46d4-4b88-a808-ed5228f2a21c admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:22.193076 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a89c0-46d4-4b88-a808-ed5228f2a21c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:22.194707 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a89c0-46d4-4b88-a808-ed5228f2a21c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:22.205677 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1256/2511] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:22.209893 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65d5b3-3a8a-4906-9384-cc9bc4636875 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:22.219993 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65d5b3-3a8a-4906-9384-cc9bc4636875 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:22.225750 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f3b84-bf2d-414f-8f38-3daa6178e8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:22.226654 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f3b84-bf2d-414f-8f38-3daa6178e8a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:22.226934 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f3b84-bf2d-414f-8f38-3daa6178e8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:22.227252 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f3b84-bf2d-414f-8f38-3daa6178e8a5 admin admin] PATH_INFO: `/v3/projects/cfebec18f91f4436b27e415ee67f8432/users/8041dc8aa088455386db88f10d3d83d6/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:22.243013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8f3b84-bf2d-414f-8f38-3daa6178e8a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfebec18f91f4436b27e415ee67f8432, user_id=8041dc8aa088455386db88f10d3d83d6, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:22.245638 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8f3b84-bf2d-414f-8f38-3daa6178e8a5 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:22.260165 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1256/2512] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:22 2022] PUT /identity/v3/projects/cfebec18f91f4436b27e415ee67f8432/users/8041dc8aa088455386db88f10d3d83d6/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:22.264285 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a89c0-46d4-4b88-a808-ed5228f2a21c admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:22.274929 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a89c0-46d4-4b88-a808-ed5228f2a21c admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:22.324988 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:22.333861 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29935ddf-8ec7-43b8-a1b1-a6c38a70a068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:22.334615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29935ddf-8ec7-43b8-a1b1-a6c38a70a068 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:22.334915 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29935ddf-8ec7-43b8-a1b1-a6c38a70a068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:22.335251 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29935ddf-8ec7-43b8-a1b1-a6c38a70a068 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:22.336186 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29935ddf-8ec7-43b8-a1b1-a6c38a70a068 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:22.338272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29935ddf-8ec7-43b8-a1b1-a6c38a70a068 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:22.349266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1257/2513] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:22.353671 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f3b84-bf2d-414f-8f38-3daa6178e8a5 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:22.364731 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f3b84-bf2d-414f-8f38-3daa6178e8a5 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:22.369350 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc0118-0929-4101-a1c7-b3cd7b49f578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:22.370236 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc0118-0929-4101-a1c7-b3cd7b49f578 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:22.370518 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc0118-0929-4101-a1c7-b3cd7b49f578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:22.370792 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc0118-0929-4101-a1c7-b3cd7b49f578 admin admin] PATH_INFO: `/v3/projects/cfebec18f91f4436b27e415ee67f8432/users/8041dc8aa088455386db88f10d3d83d6/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:22.372795 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cc0118-0929-4101-a1c7-b3cd7b49f578 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfebec18f91f4436b27e415ee67f8432, user_id=8041dc8aa088455386db88f10d3d83d6, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:22.375479 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cc0118-0929-4101-a1c7-b3cd7b49f578 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:22.385774 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1257/2514] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:22 2022] PUT /identity/v3/projects/cfebec18f91f4436b27e415ee67f8432/users/8041dc8aa088455386db88f10d3d83d6/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:22.396404 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c2988-ed8a-49e5-a934-1243c2a6d081 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:22.396664 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c2988-ed8a-49e5-a934-1243c2a6d081 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:22.396864 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c2988-ed8a-49e5-a934-1243c2a6d081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:22.425382 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-0b8c2988-ed8a-49e5-a934-1243c2a6d081 None None] MFA Rules not processed for user `8041dc8aa088455386db88f10d3d83d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:22.426614 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-0b8c2988-ed8a-49e5-a934-1243c2a6d081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:22.495156 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1258/2515] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:50:22 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:22.506497 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc0118-0929-4101-a1c7-b3cd7b49f578 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:22.508312 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-29cc0118-0929-4101-a1c7-b3cd7b49f578 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:22.549193 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:22.567118 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc0118-0929-4101-a1c7-b3cd7b49f578 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:22.570787 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b425a70b-27b8-48fe-85e1-575cc49c2d9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:22.571462 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425a70b-27b8-48fe-85e1-575cc49c2d9d service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:22.571753 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425a70b-27b8-48fe-85e1-575cc49c2d9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:22.572034 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425a70b-27b8-48fe-85e1-575cc49c2d9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:22.581484 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b425a70b-27b8-48fe-85e1-575cc49c2d9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:22.582924 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b425a70b-27b8-48fe-85e1-575cc49c2d9d service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:22.592679 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1258/2516] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:22 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:22.676213 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c2988-ed8a-49e5-a934-1243c2a6d081 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:22.685825 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c2988-ed8a-49e5-a934-1243c2a6d081 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:22.689468 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eca8ba-26e4-42b8-a0d2-e0b3a7b15f18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:22.690108 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eca8ba-26e4-42b8-a0d2-e0b3a7b15f18 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:22.690391 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eca8ba-26e4-42b8-a0d2-e0b3a7b15f18 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:22.690667 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eca8ba-26e4-42b8-a0d2-e0b3a7b15f18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:22.752456 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eca8ba-26e4-42b8-a0d2-e0b3a7b15f18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:22.755122 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eca8ba-26e4-42b8-a0d2-e0b3a7b15f18 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:22.768136 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1259/2517] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:22 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:27.431449 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b425a70b-27b8-48fe-85e1-575cc49c2d9d service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:27.494372 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b425a70b-27b8-48fe-85e1-575cc49c2d9d service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:27.498894 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94c5e95-dc1b-4dbe-9155-9f8772912cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:27.499799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c5e95-dc1b-4dbe-9155-9f8772912cf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:27.500088 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c5e95-dc1b-4dbe-9155-9f8772912cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:27.500370 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c5e95-dc1b-4dbe-9155-9f8772912cf0 admin admin] PATH_INFO: `/v3/users/30f26df76d2843768ee8456c4f2940cd` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:27.501562 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94c5e95-dc1b-4dbe-9155-9f8772912cf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30f26df76d2843768ee8456c4f2940cd)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:27.504044 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94c5e95-dc1b-4dbe-9155-9f8772912cf0 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:27.550324 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b94c5e95-dc1b-4dbe-9155-9f8772912cf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30f26df76d2843768ee8456c4f2940cd'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:27.551017 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b94c5e95-dc1b-4dbe-9155-9f8772912cf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30f26df76d2843768ee8456c4f2940cd'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:27.562851 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b94c5e95-dc1b-4dbe-9155-9f8772912cf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30f26df76d2843768ee8456c4f2940cd'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:27.571976 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b94c5e95-dc1b-4dbe-9155-9f8772912cf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30f26df76d2843768ee8456c4f2940cd'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:27.578898 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b94c5e95-dc1b-4dbe-9155-9f8772912cf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30f26df76d2843768ee8456c4f2940cd'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:27.585447 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1259/2518] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:50:27 2022] DELETE /identity/v3/users/30f26df76d2843768ee8456c4f2940cd => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:27.590367 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e849e55-f510-420b-bf79-5ad075122842 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:27.590924 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e849e55-f510-420b-bf79-5ad075122842 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:27.591063 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e849e55-f510-420b-bf79-5ad075122842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:27.608175 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-8e849e55-f510-420b-bf79-5ad075122842 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:27.609416 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-8e849e55-f510-420b-bf79-5ad075122842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:27.669017 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1260/2519] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:50:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:27.679306 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94c5e95-dc1b-4dbe-9155-9f8772912cf0 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:27.680764 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b94c5e95-dc1b-4dbe-9155-9f8772912cf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:27.738891 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94c5e95-dc1b-4dbe-9155-9f8772912cf0 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:27.742544 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0163f-23df-4c8d-aa2f-53d4f649102f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:27.743226 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0163f-23df-4c8d-aa2f-53d4f649102f service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:27.743513 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0163f-23df-4c8d-aa2f-53d4f649102f service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:27.743791 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0163f-23df-4c8d-aa2f-53d4f649102f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:27.753364 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f0163f-23df-4c8d-aa2f-53d4f649102f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:27.754786 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f0163f-23df-4c8d-aa2f-53d4f649102f service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:27.759737 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:27.764741 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1260/2520] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:28.052979 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e849e55-f510-420b-bf79-5ad075122842 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:28.055490 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-8e849e55-f510-420b-bf79-5ad075122842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:28.067957 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e849e55-f510-420b-bf79-5ad075122842 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:28.073077 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde8ecc-2a95-45c8-b730-d5035ffff416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:28.074035 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde8ecc-2a95-45c8-b730-d5035ffff416 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:28.074405 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde8ecc-2a95-45c8-b730-d5035ffff416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:28.074695 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde8ecc-2a95-45c8-b730-d5035ffff416 admin admin] PATH_INFO: `/v3/projects/ad19c4dcbcf24dc9bbc736cea10b18ac` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:28.076061 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dde8ecc-2a95-45c8-b730-d5035ffff416 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad19c4dcbcf24dc9bbc736cea10b18ac)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:28.078012 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dde8ecc-2a95-45c8-b730-d5035ffff416 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:28.090253 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:28.130772 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-0dde8ecc-2a95-45c8-b730-d5035ffff416 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad19c4dcbcf24dc9bbc736cea10b18ac', 'request_id': 'req-0dde8ecc-2a95-45c8-b730-d5035ffff416'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:28.139568 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-0dde8ecc-2a95-45c8-b730-d5035ffff416 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad19c4dcbcf24dc9bbc736cea10b18ac', 'request_id': 'req-0dde8ecc-2a95-45c8-b730-d5035ffff416'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:28.146036 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-0dde8ecc-2a95-45c8-b730-d5035ffff416 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad19c4dcbcf24dc9bbc736cea10b18ac', 'request_id': 'req-0dde8ecc-2a95-45c8-b730-d5035ffff416'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:28.156065 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-0dde8ecc-2a95-45c8-b730-d5035ffff416 admin admin] The token cache is being invalidate because project ad19c4dcbcf24dc9bbc736cea10b18ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:50:28.156348 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-0dde8ecc-2a95-45c8-b730-d5035ffff416 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:28.156991 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-0dde8ecc-2a95-45c8-b730-d5035ffff416 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:28.159104 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1261/2521] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:50:28 2022] DELETE /identity/v3/projects/ad19c4dcbcf24dc9bbc736cea10b18ac => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:28.163386 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0163f-23df-4c8d-aa2f-53d4f649102f service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:28.173046 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0163f-23df-4c8d-aa2f-53d4f649102f service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:28.229579 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66afe02-1dde-491a-af4c-4e745683c9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:28.230453 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66afe02-1dde-491a-af4c-4e745683c9fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:28.230742 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66afe02-1dde-491a-af4c-4e745683c9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:28.231053 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66afe02-1dde-491a-af4c-4e745683c9fa admin admin] PATH_INFO: `/v3/users/8041dc8aa088455386db88f10d3d83d6` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:28.232248 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66afe02-1dde-491a-af4c-4e745683c9fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=8041dc8aa088455386db88f10d3d83d6)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:28.234237 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66afe02-1dde-491a-af4c-4e745683c9fa admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:28.280042 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-f66afe02-1dde-491a-af4c-4e745683c9fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8041dc8aa088455386db88f10d3d83d6'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:28.280725 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-f66afe02-1dde-491a-af4c-4e745683c9fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8041dc8aa088455386db88f10d3d83d6'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:28.293234 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-f66afe02-1dde-491a-af4c-4e745683c9fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8041dc8aa088455386db88f10d3d83d6'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:28.301415 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-f66afe02-1dde-491a-af4c-4e745683c9fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8041dc8aa088455386db88f10d3d83d6'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:28.308265 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-f66afe02-1dde-491a-af4c-4e745683c9fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8041dc8aa088455386db88f10d3d83d6'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:28.314311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1261/2522] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:50:28 2022] DELETE /identity/v3/users/8041dc8aa088455386db88f10d3d83d6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:28.596253 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde8ecc-2a95-45c8-b730-d5035ffff416 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:28.597785 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-0dde8ecc-2a95-45c8-b730-d5035ffff416 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:28.610686 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde8ecc-2a95-45c8-b730-d5035ffff416 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:28.673471 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29784475-33bd-4b8a-b8bf-87484dc874e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:28.674770 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29784475-33bd-4b8a-b8bf-87484dc874e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:28.675083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29784475-33bd-4b8a-b8bf-87484dc874e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:28.675362 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29784475-33bd-4b8a-b8bf-87484dc874e7 admin admin] PATH_INFO: `/v3/projects/cfebec18f91f4436b27e415ee67f8432` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:28.676595 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29784475-33bd-4b8a-b8bf-87484dc874e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfebec18f91f4436b27e415ee67f8432)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:28.678516 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29784475-33bd-4b8a-b8bf-87484dc874e7 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:28.725381 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-29784475-33bd-4b8a-b8bf-87484dc874e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfebec18f91f4436b27e415ee67f8432', 'request_id': 'req-29784475-33bd-4b8a-b8bf-87484dc874e7'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:28.733288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-29784475-33bd-4b8a-b8bf-87484dc874e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfebec18f91f4436b27e415ee67f8432', 'request_id': 'req-29784475-33bd-4b8a-b8bf-87484dc874e7'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:28.740334 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-29784475-33bd-4b8a-b8bf-87484dc874e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfebec18f91f4436b27e415ee67f8432', 'request_id': 'req-29784475-33bd-4b8a-b8bf-87484dc874e7'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:28.750772 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-29784475-33bd-4b8a-b8bf-87484dc874e7 admin admin] The token cache is being invalidate because project cfebec18f91f4436b27e415ee67f8432 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:50:28.751153 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-29784475-33bd-4b8a-b8bf-87484dc874e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:28.751924 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-29784475-33bd-4b8a-b8bf-87484dc874e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:28.754001 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1262/2523] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:50:28 2022] DELETE /identity/v3/projects/cfebec18f91f4436b27e415ee67f8432 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:28.758872 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296b02d-0417-4b5a-9f1a-d7d9b1fdc2db None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:28.759497 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296b02d-0417-4b5a-9f1a-d7d9b1fdc2db None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:28.759497 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296b02d-0417-4b5a-9f1a-d7d9b1fdc2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:28.777335 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-8296b02d-0417-4b5a-9f1a-d7d9b1fdc2db None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:28.778525 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-8296b02d-0417-4b5a-9f1a-d7d9b1fdc2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:28.840717 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1262/2524] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:50:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:28.849972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184eb08-e84c-4cef-a17f-1767962f15dd None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:28.849972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184eb08-e84c-4cef-a17f-1767962f15dd None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:28.850233 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184eb08-e84c-4cef-a17f-1767962f15dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:28.868794 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-c184eb08-e84c-4cef-a17f-1767962f15dd None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:28.870015 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c184eb08-e84c-4cef-a17f-1767962f15dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:28.878058 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1263/2525] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:50:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:28.882601 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8296b02d-0417-4b5a-9f1a-d7d9b1fdc2db None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:28.892404 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8296b02d-0417-4b5a-9f1a-d7d9b1fdc2db None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:28.897126 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181d9ca-1035-4ea2-83da-5582ae65c962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:28.897788 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181d9ca-1035-4ea2-83da-5582ae65c962 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:28.898071 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181d9ca-1035-4ea2-83da-5582ae65c962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:28.898349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181d9ca-1035-4ea2-83da-5582ae65c962 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:28.899162 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181d9ca-1035-4ea2-83da-5582ae65c962 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:50:28.899474 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181d9ca-1035-4ea2-83da-5582ae65c962 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:28.901077 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181d9ca-1035-4ea2-83da-5582ae65c962 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:28.913654 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1263/2526] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:50:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:50:28.917963 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c184eb08-e84c-4cef-a17f-1767962f15dd None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:28.927916 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c184eb08-e84c-4cef-a17f-1767962f15dd None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:28.932781 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f20602b-8656-4522-9483-535dcd37695a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:28.933556 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20602b-8656-4522-9483-535dcd37695a admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:28.933920 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20602b-8656-4522-9483-535dcd37695a admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:28.934200 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20602b-8656-4522-9483-535dcd37695a admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:28.934893 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f20602b-8656-4522-9483-535dcd37695a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:28.936879 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f20602b-8656-4522-9483-535dcd37695a admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:28.956575 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:28.967505 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1264/2527] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:50:28 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:28.968609 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181d9ca-1035-4ea2-83da-5582ae65c962 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:28.979511 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181d9ca-1035-4ea2-83da-5582ae65c962 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:28.983350 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:29.038814 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760bceb9-f5fb-4b97-8a6f-313f9fedf697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:29.039731 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760bceb9-f5fb-4b97-8a6f-313f9fedf697 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:29.040017 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760bceb9-f5fb-4b97-8a6f-313f9fedf697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:29.040291 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760bceb9-f5fb-4b97-8a6f-313f9fedf697 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:29.040979 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760bceb9-f5fb-4b97-8a6f-313f9fedf697 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:29.042876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760bceb9-f5fb-4b97-8a6f-313f9fedf697 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:29.070301 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1264/2528] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:50:28 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:50:29.074508 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f20602b-8656-4522-9483-535dcd37695a admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:29.085859 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f20602b-8656-4522-9483-535dcd37695a admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:29.090636 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf52b81-d5e7-4a01-9738-fca076c565b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:29.091350 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf52b81-d5e7-4a01-9738-fca076c565b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:29.091638 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf52b81-d5e7-4a01-9738-fca076c565b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:29.091916 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf52b81-d5e7-4a01-9738-fca076c565b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:29.092650 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf52b81-d5e7-4a01-9738-fca076c565b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:29.094259 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf52b81-d5e7-4a01-9738-fca076c565b1 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:29.105825 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1265/2529] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:29.110133 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760bceb9-f5fb-4b97-8a6f-313f9fedf697 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:29.120687 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760bceb9-f5fb-4b97-8a6f-313f9fedf697 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:29.125317 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6820c23-67e3-445e-9988-ad73eae9ee91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:29.125972 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6820c23-67e3-445e-9988-ad73eae9ee91 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:29.126257 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6820c23-67e3-445e-9988-ad73eae9ee91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:29.126534 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6820c23-67e3-445e-9988-ad73eae9ee91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:29.127297 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6820c23-67e3-445e-9988-ad73eae9ee91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:29.128903 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6820c23-67e3-445e-9988-ad73eae9ee91 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:29.138915 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1265/2530] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:29.144185 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf52b81-d5e7-4a01-9738-fca076c565b1 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:29.153508 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf52b81-d5e7-4a01-9738-fca076c565b1 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:29.158848 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c5e0d-90fa-4da9-90c0-c5d56d17a559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:29.159763 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c5e0d-90fa-4da9-90c0-c5d56d17a559 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:29.160052 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c5e0d-90fa-4da9-90c0-c5d56d17a559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:29.160326 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c5e0d-90fa-4da9-90c0-c5d56d17a559 admin admin] PATH_INFO: `/v3/projects/adc3fee639bd43c1b1b09f91b7d26ce3/users/8757e4bdf31c4e96a50f8484d18872b5/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:29.175752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c5e0d-90fa-4da9-90c0-c5d56d17a559 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adc3fee639bd43c1b1b09f91b7d26ce3, user_id=8757e4bdf31c4e96a50f8484d18872b5, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:29.178483 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c5e0d-90fa-4da9-90c0-c5d56d17a559 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:29.191819 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1266/2531] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:29 2022] PUT /identity/v3/projects/adc3fee639bd43c1b1b09f91b7d26ce3/users/8757e4bdf31c4e96a50f8484d18872b5/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:29.196876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bd022-6f0c-4b7e-8e86-42d39a80b650 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:29.197192 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bd022-6f0c-4b7e-8e86-42d39a80b650 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:29.197502 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bd022-6f0c-4b7e-8e86-42d39a80b650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:29.216272 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-2b6bd022-6f0c-4b7e-8e86-42d39a80b650 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:29.217481 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-2b6bd022-6f0c-4b7e-8e86-42d39a80b650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:29.276227 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1266/2532] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:50:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:29.287659 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c5e0d-90fa-4da9-90c0-c5d56d17a559 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:29.289403 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5f1c5e0d-90fa-4da9-90c0-c5d56d17a559 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:29.346654 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c5e0d-90fa-4da9-90c0-c5d56d17a559 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:29.350354 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22dc1e7-abcb-445f-993a-df46de9caa65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:29.351024 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22dc1e7-abcb-445f-993a-df46de9caa65 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:29.351320 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22dc1e7-abcb-445f-993a-df46de9caa65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:29.351598 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22dc1e7-abcb-445f-993a-df46de9caa65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:29.360822 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22dc1e7-abcb-445f-993a-df46de9caa65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:29.362271 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22dc1e7-abcb-445f-993a-df46de9caa65 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:29.371927 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1267/2533] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:50:30.010455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca98a3-5b74-4fa6-9f32-e0fb06c9e673 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:30.011016 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca98a3-5b74-4fa6-9f32-e0fb06c9e673 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:30.011391 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca98a3-5b74-4fa6-9f32-e0fb06c9e673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:30.040366 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-aeca98a3-5b74-4fa6-9f32-e0fb06c9e673 None None] MFA Rules not processed for user `8757e4bdf31c4e96a50f8484d18872b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:30.041598 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-aeca98a3-5b74-4fa6-9f32-e0fb06c9e673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:30.108807 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1267/2534] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:50:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:30.120685 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22dc1e7-abcb-445f-993a-df46de9caa65 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:30.122207 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-b22dc1e7-abcb-445f-993a-df46de9caa65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:30.186333 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22dc1e7-abcb-445f-993a-df46de9caa65 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:30.190818 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16750e-bc53-43c1-9a3f-2e13a13db9b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:30.191536 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16750e-bc53-43c1-9a3f-2e13a13db9b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:30.191826 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16750e-bc53-43c1-9a3f-2e13a13db9b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:30.192106 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16750e-bc53-43c1-9a3f-2e13a13db9b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:30.201736 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16750e-bc53-43c1-9a3f-2e13a13db9b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:30.203249 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16750e-bc53-43c1-9a3f-2e13a13db9b5 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:30.213978 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1268/2535] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:31.567457 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f285a-f0c1-4d16-a69a-bcd3882885ea None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:31.568022 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f285a-f0c1-4d16-a69a-bcd3882885ea None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:31.568022 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f285a-f0c1-4d16-a69a-bcd3882885ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:31.585556 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-a93f285a-f0c1-4d16-a69a-bcd3882885ea None None] MFA Rules not processed for user `8757e4bdf31c4e96a50f8484d18872b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:31.586667 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-a93f285a-f0c1-4d16-a69a-bcd3882885ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:31.594080 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1268/2536] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:50:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:40.209278 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=67,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:41.580244 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=93,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:47.056530 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3af3c-e094-4a46-b8e2-93ebe0342b42 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:47.057010 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3af3c-e094-4a46-b8e2-93ebe0342b42 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:47.057120 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3af3c-e094-4a46-b8e2-93ebe0342b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:47.074477 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-c7a3af3c-e094-4a46-b8e2-93ebe0342b42 None None] MFA Rules not processed for user `8757e4bdf31c4e96a50f8484d18872b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:47.075732 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c7a3af3c-e094-4a46-b8e2-93ebe0342b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:47.082613 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1269/2537] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:50:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:57.069640 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:03.387962 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f285a-f0c1-4d16-a69a-bcd3882885ea None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:03.389379 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-a93f285a-f0c1-4d16-a69a-bcd3882885ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:03.459425 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f285a-f0c1-4d16-a69a-bcd3882885ea None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:03.464358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d477310-3344-458e-9e5f-a5d342bb6a01 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4128111a9aa450abb2146a5d9926dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:03.465015 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d477310-3344-458e-9e5f-a5d342bb6a01 service placement] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:03.465301 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d477310-3344-458e-9e5f-a5d342bb6a01 service placement] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:03.465577 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d477310-3344-458e-9e5f-a5d342bb6a01 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:03.467347 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-6d477310-3344-458e-9e5f-a5d342bb6a01 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:03.479141 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d477310-3344-458e-9e5f-a5d342bb6a01 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:03.480706 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d477310-3344-458e-9e5f-a5d342bb6a01 service placement] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:03.490261 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1269/2538] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:11.473614 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7c0aa-b40b-487f-abd9-029b5eb8c0df None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:11.474128 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7c0aa-b40b-487f-abd9-029b5eb8c0df None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:11.474233 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7c0aa-b40b-487f-abd9-029b5eb8c0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:11.491977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-f6e7c0aa-b40b-487f-abd9-029b5eb8c0df None None] MFA Rules not processed for user `bf393488726e49338e70017f5cb7c4be`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:11.493185 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f6e7c0aa-b40b-487f-abd9-029b5eb8c0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:11.549724 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1270/2539] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Jun 15 14:51:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:11.598024 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d477310-3344-458e-9e5f-a5d342bb6a01 service placement] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:11.599605 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-6d477310-3344-458e-9e5f-a5d342bb6a01 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:11.669494 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d477310-3344-458e-9e5f-a5d342bb6a01 service placement] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:11.674525 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11085a2c-5919-4d2d-ae8c-7ee52581f08b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '641a81b3140f47119b5c2323a97a6681', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:11.675200 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11085a2c-5919-4d2d-ae8c-7ee52581f08b service swift] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:11.675485 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11085a2c-5919-4d2d-ae8c-7ee52581f08b service swift] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:11.675767 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11085a2c-5919-4d2d-ae8c-7ee52581f08b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:11.684458 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11085a2c-5919-4d2d-ae8c-7ee52581f08b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:11.686059 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11085a2c-5919-4d2d-ae8c-7ee52581f08b service swift] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:11.694876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1270/2540] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Jun 15 14:51:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:51:21.542915 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=25 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:21.691986 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=64 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:27.111753 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26417a1-f31c-4d61-86a7-2c8ee17c8a60 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:27.112203 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26417a1-f31c-4d61-86a7-2c8ee17c8a60 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:27.112307 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26417a1-f31c-4d61-86a7-2c8ee17c8a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:27.131823 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-e26417a1-f31c-4d61-86a7-2c8ee17c8a60 None None] MFA Rules not processed for user `bf393488726e49338e70017f5cb7c4be`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:27.133025 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e26417a1-f31c-4d61-86a7-2c8ee17c8a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:27.140181 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1271/2541] 10.222.0.45 () {58 vars in 967 bytes} [Wed Jun 15 14:51:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:27.152648 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11085a2c-5919-4d2d-ae8c-7ee52581f08b service swift] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:27.163480 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11085a2c-5919-4d2d-ae8c-7ee52581f08b service swift] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:27.168507 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5374830f-e613-4226-a3ab-bd1b1e6fd55c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '641a81b3140f47119b5c2323a97a6681', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:27.169147 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374830f-e613-4226-a3ab-bd1b1e6fd55c service swift] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:27.169435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374830f-e613-4226-a3ab-bd1b1e6fd55c service swift] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:27.169718 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374830f-e613-4226-a3ab-bd1b1e6fd55c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:27.178871 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5374830f-e613-4226-a3ab-bd1b1e6fd55c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:27.180504 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5374830f-e613-4226-a3ab-bd1b1e6fd55c service swift] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:27.189669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1271/2542] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Jun 15 14:51:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:51:29.176906 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26417a1-f31c-4d61-86a7-2c8ee17c8a60 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:29.188136 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26417a1-f31c-4d61-86a7-2c8ee17c8a60 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:29.192732 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acde100-adcf-4c6e-bbf9-208ca473b0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:29.193594 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acde100-adcf-4c6e-bbf9-208ca473b0ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:29.193872 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acde100-adcf-4c6e-bbf9-208ca473b0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:29.194145 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acde100-adcf-4c6e-bbf9-208ca473b0ce admin admin] PATH_INFO: `/v3/users/8757e4bdf31c4e96a50f8484d18872b5` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:29.195361 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acde100-adcf-4c6e-bbf9-208ca473b0ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=8757e4bdf31c4e96a50f8484d18872b5)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:29.197276 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acde100-adcf-4c6e-bbf9-208ca473b0ce admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:29.245526 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-5acde100-adcf-4c6e-bbf9-208ca473b0ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8757e4bdf31c4e96a50f8484d18872b5'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:29.246177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-5acde100-adcf-4c6e-bbf9-208ca473b0ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8757e4bdf31c4e96a50f8484d18872b5'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:29.259268 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-5acde100-adcf-4c6e-bbf9-208ca473b0ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8757e4bdf31c4e96a50f8484d18872b5'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:29.259981 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-5acde100-adcf-4c6e-bbf9-208ca473b0ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8757e4bdf31c4e96a50f8484d18872b5'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:29.268411 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-5acde100-adcf-4c6e-bbf9-208ca473b0ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8757e4bdf31c4e96a50f8484d18872b5'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:29.280740 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1272/2543] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:51:29 2022] DELETE /identity/v3/users/8757e4bdf31c4e96a50f8484d18872b5 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:29.457919 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5374830f-e613-4226-a3ab-bd1b1e6fd55c service swift] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:29.469287 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5374830f-e613-4226-a3ab-bd1b1e6fd55c service swift] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:29.530550 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b1021-8f69-4a1d-b39d-a15ecc289911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:29.531439 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b1021-8f69-4a1d-b39d-a15ecc289911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:29.531892 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b1021-8f69-4a1d-b39d-a15ecc289911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:29.532911 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b1021-8f69-4a1d-b39d-a15ecc289911 admin admin] PATH_INFO: `/v3/projects/adc3fee639bd43c1b1b09f91b7d26ce3` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:29.534491 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00b1021-8f69-4a1d-b39d-a15ecc289911 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adc3fee639bd43c1b1b09f91b7d26ce3)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:29.536448 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00b1021-8f69-4a1d-b39d-a15ecc289911 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:29.583545 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a00b1021-8f69-4a1d-b39d-a15ecc289911 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adc3fee639bd43c1b1b09f91b7d26ce3', 'request_id': 'req-a00b1021-8f69-4a1d-b39d-a15ecc289911'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:29.590761 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a00b1021-8f69-4a1d-b39d-a15ecc289911 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adc3fee639bd43c1b1b09f91b7d26ce3', 'request_id': 'req-a00b1021-8f69-4a1d-b39d-a15ecc289911'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:29.598322 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a00b1021-8f69-4a1d-b39d-a15ecc289911 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adc3fee639bd43c1b1b09f91b7d26ce3', 'request_id': 'req-a00b1021-8f69-4a1d-b39d-a15ecc289911'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:29.609168 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a00b1021-8f69-4a1d-b39d-a15ecc289911 admin admin] The token cache is being invalidate because project adc3fee639bd43c1b1b09f91b7d26ce3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:51:29.609458 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a00b1021-8f69-4a1d-b39d-a15ecc289911 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:29.610135 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a00b1021-8f69-4a1d-b39d-a15ecc289911 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:29.611725 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1272/2544] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:51:29 2022] DELETE /identity/v3/projects/adc3fee639bd43c1b1b09f91b7d26ce3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:29.616738 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0ecb1-3534-4cf9-ae6f-6a0d3d6e41b8 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:29.617030 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0ecb1-3534-4cf9-ae6f-6a0d3d6e41b8 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:29.617310 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0ecb1-3534-4cf9-ae6f-6a0d3d6e41b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:29.635531 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-50a0ecb1-3534-4cf9-ae6f-6a0d3d6e41b8 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:29.636653 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-50a0ecb1-3534-4cf9-ae6f-6a0d3d6e41b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:29.696148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1273/2545] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:51:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:29.704654 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2e349-5d0e-4e21-bb84-75a1e420dff0 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:29.704948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2e349-5d0e-4e21-bb84-75a1e420dff0 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:29.705233 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2e349-5d0e-4e21-bb84-75a1e420dff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:29.723520 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-74b2e349-5d0e-4e21-bb84-75a1e420dff0 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:29.724729 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-74b2e349-5d0e-4e21-bb84-75a1e420dff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:29.732813 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1273/2546] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:51:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:29.737130 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a0ecb1-3534-4cf9-ae6f-6a0d3d6e41b8 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:29.747038 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a0ecb1-3534-4cf9-ae6f-6a0d3d6e41b8 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:29.752171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205cea2a-5366-4933-9882-87ed0b186f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:29.752824 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205cea2a-5366-4933-9882-87ed0b186f71 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:29.753107 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205cea2a-5366-4933-9882-87ed0b186f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:29.753380 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205cea2a-5366-4933-9882-87ed0b186f71 admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:29.754137 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205cea2a-5366-4933-9882-87ed0b186f71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:51:29.754442 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205cea2a-5366-4933-9882-87ed0b186f71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:29.756082 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205cea2a-5366-4933-9882-87ed0b186f71 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:29.768371 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1274/2547] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:51:29.772554 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2e349-5d0e-4e21-bb84-75a1e420dff0 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:29.782012 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2e349-5d0e-4e21-bb84-75a1e420dff0 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:29.786650 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc50001-33f2-46c2-a2a0-bf2b0f8b0696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:29.787600 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc50001-33f2-46c2-a2a0-bf2b0f8b0696 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:29.787876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc50001-33f2-46c2-a2a0-bf2b0f8b0696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:29.788146 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc50001-33f2-46c2-a2a0-bf2b0f8b0696 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:29.788828 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc50001-33f2-46c2-a2a0-bf2b0f8b0696 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:29.790708 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc50001-33f2-46c2-a2a0-bf2b0f8b0696 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:29.821042 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1274/2548] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:51:29 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:29.825127 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205cea2a-5366-4933-9882-87ed0b186f71 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:29.835717 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205cea2a-5366-4933-9882-87ed0b186f71 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:29.884871 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:29.897000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9d4e0-3417-4653-839a-e3694183daef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:29.898181 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9d4e0-3417-4653-839a-e3694183daef admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:29.898647 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9d4e0-3417-4653-839a-e3694183daef admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:29.899177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9d4e0-3417-4653-839a-e3694183daef admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:29.901543 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c9d4e0-3417-4653-839a-e3694183daef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:29.902494 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c9d4e0-3417-4653-839a-e3694183daef admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:29.924332 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1275/2549] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:51:29 2022] POST /identity/v3/users => generated 285 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:29.928538 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc50001-33f2-46c2-a2a0-bf2b0f8b0696 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:29.939575 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc50001-33f2-46c2-a2a0-bf2b0f8b0696 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:29.944474 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38632e4-c9af-4770-b4d2-d8f76f149e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:29.945151 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38632e4-c9af-4770-b4d2-d8f76f149e5a admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:29.945443 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38632e4-c9af-4770-b4d2-d8f76f149e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:29.945759 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38632e4-c9af-4770-b4d2-d8f76f149e5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:29.946493 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38632e4-c9af-4770-b4d2-d8f76f149e5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:29.948214 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38632e4-c9af-4770-b4d2-d8f76f149e5a admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:29.961335 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1275/2550] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:51:29.965516 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9d4e0-3417-4653-839a-e3694183daef admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:29.976003 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9d4e0-3417-4653-839a-e3694183daef admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:29.980815 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4d51a0-0972-404e-89bf-29d3d01174f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:29.981487 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d51a0-0972-404e-89bf-29d3d01174f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:29.981784 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d51a0-0972-404e-89bf-29d3d01174f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:29.982062 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d51a0-0972-404e-89bf-29d3d01174f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:29.982797 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4d51a0-0972-404e-89bf-29d3d01174f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:29.984817 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4d51a0-0972-404e-89bf-29d3d01174f9 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:29.995653 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1276/2551] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:51:29.999865 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38632e4-c9af-4770-b4d2-d8f76f149e5a admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:30.010357 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38632e4-c9af-4770-b4d2-d8f76f149e5a admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:30.015052 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309adcd3-7833-4f9e-ab19-a89c7877998b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:30.015940 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309adcd3-7833-4f9e-ab19-a89c7877998b admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:30.016336 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309adcd3-7833-4f9e-ab19-a89c7877998b admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:30.016601 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309adcd3-7833-4f9e-ab19-a89c7877998b admin admin] PATH_INFO: `/v3/projects/af085f0f0b8945bd9ec5aac95af67955/users/bc1a5ba5dbfa404dad6aacc16fa3993c/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:30.035640 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309adcd3-7833-4f9e-ab19-a89c7877998b admin admin] RBAC: Authorizing `identity:create_grant(project_id=af085f0f0b8945bd9ec5aac95af67955, user_id=bc1a5ba5dbfa404dad6aacc16fa3993c, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:30.038840 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309adcd3-7833-4f9e-ab19-a89c7877998b admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:30.056418 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1276/2552] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:29 2022] PUT /identity/v3/projects/af085f0f0b8945bd9ec5aac95af67955/users/bc1a5ba5dbfa404dad6aacc16fa3993c/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:51:30.067498 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950031d-f405-43f3-9ba9-c2532a709c7c None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:30.067793 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950031d-f405-43f3-9ba9-c2532a709c7c None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:30.068077 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950031d-f405-43f3-9ba9-c2532a709c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:30.094717 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-2950031d-f405-43f3-9ba9-c2532a709c7c None None] MFA Rules not processed for user `bc1a5ba5dbfa404dad6aacc16fa3993c`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:30.095993 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-2950031d-f405-43f3-9ba9-c2532a709c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:30.162640 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1277/2553] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:51:30 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:30.174814 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309adcd3-7833-4f9e-ab19-a89c7877998b admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:30.176826 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-309adcd3-7833-4f9e-ab19-a89c7877998b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:30.191875 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:30.237317 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309adcd3-7833-4f9e-ab19-a89c7877998b admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:30.240988 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99974b-008b-40ed-a716-a7ff66cbcc38 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f10214b2bb4eafb3dff7adf71b9bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:30.241640 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99974b-008b-40ed-a716-a7ff66cbcc38 service glance] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:30.241928 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99974b-008b-40ed-a716-a7ff66cbcc38 service glance] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:30.242211 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99974b-008b-40ed-a716-a7ff66cbcc38 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:30.250518 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a99974b-008b-40ed-a716-a7ff66cbcc38 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:30.252734 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a99974b-008b-40ed-a716-a7ff66cbcc38 service glance] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:30.261939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1277/2554] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:30 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:37.658650 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb1147-fcbc-4560-bf5a-7833039c49de None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:37.659200 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb1147-fcbc-4560-bf5a-7833039c49de None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:37.659305 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb1147-fcbc-4560-bf5a-7833039c49de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:37.677863 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-b2eb1147-fcbc-4560-bf5a-7833039c49de None None] MFA Rules not processed for user `bf393488726e49338e70017f5cb7c4be`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:37.679117 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-b2eb1147-fcbc-4560-bf5a-7833039c49de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:37.737634 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1278/2555] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Jun 15 14:51:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:37.750685 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99974b-008b-40ed-a716-a7ff66cbcc38 service glance] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:37.752148 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-2a99974b-008b-40ed-a716-a7ff66cbcc38 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:37.824007 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99974b-008b-40ed-a716-a7ff66cbcc38 service glance] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:37.829816 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e718ceb-64c1-4757-a036-8bfa8ba993ff service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '641a81b3140f47119b5c2323a97a6681', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:37.830465 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e718ceb-64c1-4757-a036-8bfa8ba993ff service swift] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:37.830751 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e718ceb-64c1-4757-a036-8bfa8ba993ff service swift] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:37.831053 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e718ceb-64c1-4757-a036-8bfa8ba993ff service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:37.839207 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e718ceb-64c1-4757-a036-8bfa8ba993ff service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:37.841422 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e718ceb-64c1-4757-a036-8bfa8ba993ff service swift] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:37.852232 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1278/2556] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Jun 15 14:51:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:51:41.439168 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb1147-fcbc-4560-bf5a-7833039c49de None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:41.440640 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-b2eb1147-fcbc-4560-bf5a-7833039c49de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:41.501955 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb1147-fcbc-4560-bf5a-7833039c49de None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:41.505641 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d21cd7a-3145-4dc3-87d4-3e45b9bfecac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:41.506289 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21cd7a-3145-4dc3-87d4-3e45b9bfecac service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:41.506723 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21cd7a-3145-4dc3-87d4-3e45b9bfecac service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:41.507020 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21cd7a-3145-4dc3-87d4-3e45b9bfecac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:41.508919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-0d21cd7a-3145-4dc3-87d4-3e45b9bfecac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:41.531067 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:41.576133 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d21cd7a-3145-4dc3-87d4-3e45b9bfecac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:41.577855 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d21cd7a-3145-4dc3-87d4-3e45b9bfecac service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:41.587734 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1279/2557] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:47.848684 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=52 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:48.348505 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e718ceb-64c1-4757-a036-8bfa8ba993ff service swift] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:48.350175 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1e718ceb-64c1-4757-a036-8bfa8ba993ff service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:48.375185 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e718ceb-64c1-4757-a036-8bfa8ba993ff service swift] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:48.380569 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4743d-fff5-4576-8c15-91f99622570e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:48.381372 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4743d-fff5-4576-8c15-91f99622570e service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:48.381682 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4743d-fff5-4576-8c15-91f99622570e service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:48.381956 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4743d-fff5-4576-8c15-91f99622570e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:48.390064 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-dda4743d-fff5-4576-8c15-91f99622570e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:48.499890 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda4743d-fff5-4576-8c15-91f99622570e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:48.502161 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda4743d-fff5-4576-8c15-91f99622570e service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:48.513937 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1279/2558] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:51.583839 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=18 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:52.828122 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d21cd7a-3145-4dc3-87d4-3e45b9bfecac service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:52.830239 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-0d21cd7a-3145-4dc3-87d4-3e45b9bfecac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:52.922718 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d21cd7a-3145-4dc3-87d4-3e45b9bfecac service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:52.927935 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074b81b0-bc06-42fe-9a22-dd88ebce54d8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4128111a9aa450abb2146a5d9926dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:52.928603 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b81b0-bc06-42fe-9a22-dd88ebce54d8 service placement] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:52.928891 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b81b0-bc06-42fe-9a22-dd88ebce54d8 service placement] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:52.929171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b81b0-bc06-42fe-9a22-dd88ebce54d8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:52.931016 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-074b81b0-bc06-42fe-9a22-dd88ebce54d8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:52.943843 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074b81b0-bc06-42fe-9a22-dd88ebce54d8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:52.945473 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074b81b0-bc06-42fe-9a22-dd88ebce54d8 service placement] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:52.956259 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1280/2559] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:56.544539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4743d-fff5-4576-8c15-91f99622570e service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:56.545710 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-dda4743d-fff5-4576-8c15-91f99622570e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:56.567192 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4743d-fff5-4576-8c15-91f99622570e service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:56.577771 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e64723-0637-415d-9546-0bec051c0741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:56.577771 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e64723-0637-415d-9546-0bec051c0741 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:56.577771 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e64723-0637-415d-9546-0bec051c0741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:56.577771 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e64723-0637-415d-9546-0bec051c0741 admin admin] PATH_INFO: `/v3/users/5d47f5d89bbb4d43844b1fb0794a3e69` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:56.579099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e64723-0637-415d-9546-0bec051c0741 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d47f5d89bbb4d43844b1fb0794a3e69)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:56.581518 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e64723-0637-415d-9546-0bec051c0741 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:56.654801 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-83e64723-0637-415d-9546-0bec051c0741 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d47f5d89bbb4d43844b1fb0794a3e69'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:56.655762 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-83e64723-0637-415d-9546-0bec051c0741 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d47f5d89bbb4d43844b1fb0794a3e69'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:56.671137 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-83e64723-0637-415d-9546-0bec051c0741 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d47f5d89bbb4d43844b1fb0794a3e69'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:56.694445 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-83e64723-0637-415d-9546-0bec051c0741 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d47f5d89bbb4d43844b1fb0794a3e69'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:56.704497 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-83e64723-0637-415d-9546-0bec051c0741 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d47f5d89bbb4d43844b1fb0794a3e69'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:56.711227 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1280/2560] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:51:56 2022] DELETE /identity/v3/users/5d47f5d89bbb4d43844b1fb0794a3e69 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:56.898690 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074b81b0-bc06-42fe-9a22-dd88ebce54d8 service placement] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:56.911703 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074b81b0-bc06-42fe-9a22-dd88ebce54d8 service placement] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:56.977606 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566c178-3fae-433b-9655-ec5ec6723afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:56.978469 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566c178-3fae-433b-9655-ec5ec6723afa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:56.978756 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566c178-3fae-433b-9655-ec5ec6723afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:56.979064 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566c178-3fae-433b-9655-ec5ec6723afa admin admin] PATH_INFO: `/v3/projects/3e38db34f53b4271ab5462a872e954ef` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:56.980314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3566c178-3fae-433b-9655-ec5ec6723afa admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e38db34f53b4271ab5462a872e954ef)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:56.982239 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3566c178-3fae-433b-9655-ec5ec6723afa admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:57.031061 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3566c178-3fae-433b-9655-ec5ec6723afa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e38db34f53b4271ab5462a872e954ef', 'request_id': 'req-3566c178-3fae-433b-9655-ec5ec6723afa'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:57.039168 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3566c178-3fae-433b-9655-ec5ec6723afa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e38db34f53b4271ab5462a872e954ef', 'request_id': 'req-3566c178-3fae-433b-9655-ec5ec6723afa'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:57.045408 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3566c178-3fae-433b-9655-ec5ec6723afa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e38db34f53b4271ab5462a872e954ef', 'request_id': 'req-3566c178-3fae-433b-9655-ec5ec6723afa'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:57.056239 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3566c178-3fae-433b-9655-ec5ec6723afa admin admin] The token cache is being invalidate because project 3e38db34f53b4271ab5462a872e954ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:51:57.056535 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3566c178-3fae-433b-9655-ec5ec6723afa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:57.057188 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3566c178-3fae-433b-9655-ec5ec6723afa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:57.059043 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1281/2561] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:51:56 2022] DELETE /identity/v3/projects/3e38db34f53b4271ab5462a872e954ef => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:57.063164 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e64723-0637-415d-9546-0bec051c0741 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:57.064601 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-83e64723-0637-415d-9546-0bec051c0741 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:57.077880 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e64723-0637-415d-9546-0bec051c0741 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:57.134131 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a53b8f-b39f-451d-afd6-ddc4704f2520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:57.135022 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a53b8f-b39f-451d-afd6-ddc4704f2520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:57.135313 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a53b8f-b39f-451d-afd6-ddc4704f2520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:57.135595 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a53b8f-b39f-451d-afd6-ddc4704f2520 admin admin] PATH_INFO: `/v3/users/0d0af5aca957486ea840018795e438c5` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:57.136787 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a53b8f-b39f-451d-afd6-ddc4704f2520 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d0af5aca957486ea840018795e438c5)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:57.138710 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a53b8f-b39f-451d-afd6-ddc4704f2520 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:57.144514 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:57.187914 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c4a53b8f-b39f-451d-afd6-ddc4704f2520 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d0af5aca957486ea840018795e438c5'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:57.188260 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c4a53b8f-b39f-451d-afd6-ddc4704f2520 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d0af5aca957486ea840018795e438c5'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:57.201115 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c4a53b8f-b39f-451d-afd6-ddc4704f2520 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d0af5aca957486ea840018795e438c5'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:57.211074 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c4a53b8f-b39f-451d-afd6-ddc4704f2520 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d0af5aca957486ea840018795e438c5'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:57.218257 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c4a53b8f-b39f-451d-afd6-ddc4704f2520 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d0af5aca957486ea840018795e438c5'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:57.228828 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1281/2562] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:51:57 2022] DELETE /identity/v3/users/0d0af5aca957486ea840018795e438c5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:57.454145 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566c178-3fae-433b-9655-ec5ec6723afa admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:57.455778 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-3566c178-3fae-433b-9655-ec5ec6723afa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:57.470587 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566c178-3fae-433b-9655-ec5ec6723afa admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:57.491510 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:57.542080 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57be3e7-980a-4f78-a3b1-69fa15a6d7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:57.543749 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57be3e7-980a-4f78-a3b1-69fa15a6d7c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:57.544072 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57be3e7-980a-4f78-a3b1-69fa15a6d7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:57.544389 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57be3e7-980a-4f78-a3b1-69fa15a6d7c0 admin admin] PATH_INFO: `/v3/projects/f5189f66322f4fb6a6e85bfdfca8b4e2` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:57.545690 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57be3e7-980a-4f78-a3b1-69fa15a6d7c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5189f66322f4fb6a6e85bfdfca8b4e2)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:57.548192 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57be3e7-980a-4f78-a3b1-69fa15a6d7c0 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:57.604751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b57be3e7-980a-4f78-a3b1-69fa15a6d7c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5189f66322f4fb6a6e85bfdfca8b4e2', 'request_id': 'req-b57be3e7-980a-4f78-a3b1-69fa15a6d7c0'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:57.613554 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b57be3e7-980a-4f78-a3b1-69fa15a6d7c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5189f66322f4fb6a6e85bfdfca8b4e2', 'request_id': 'req-b57be3e7-980a-4f78-a3b1-69fa15a6d7c0'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:57.620448 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b57be3e7-980a-4f78-a3b1-69fa15a6d7c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5189f66322f4fb6a6e85bfdfca8b4e2', 'request_id': 'req-b57be3e7-980a-4f78-a3b1-69fa15a6d7c0'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:57.632607 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b57be3e7-980a-4f78-a3b1-69fa15a6d7c0 admin admin] The token cache is being invalidate because project f5189f66322f4fb6a6e85bfdfca8b4e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:51:57.632893 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b57be3e7-980a-4f78-a3b1-69fa15a6d7c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:57.633559 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b57be3e7-980a-4f78-a3b1-69fa15a6d7c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:57.635138 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1282/2563] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:51:57 2022] DELETE /identity/v3/projects/f5189f66322f4fb6a6e85bfdfca8b4e2 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:03.560233 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939cfd5-5446-4172-9152-32ef3e4aae9e None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:03.560716 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939cfd5-5446-4172-9152-32ef3e4aae9e None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:03.560810 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939cfd5-5446-4172-9152-32ef3e4aae9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:03.578971 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-7939cfd5-5446-4172-9152-32ef3e4aae9e None None] MFA Rules not processed for user `bf393488726e49338e70017f5cb7c4be`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:03.580225 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-7939cfd5-5446-4172-9152-32ef3e4aae9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:03.652484 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1282/2564] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Jun 15 14:52:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:03.667280 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57be3e7-980a-4f78-a3b1-69fa15a6d7c0 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:03.668976 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-b57be3e7-980a-4f78-a3b1-69fa15a6d7c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:03.758571 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57be3e7-980a-4f78-a3b1-69fa15a6d7c0 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:03.763716 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab3df5-8522-4275-adf6-faad82b0fc72 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '641a81b3140f47119b5c2323a97a6681', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:03.764373 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab3df5-8522-4275-adf6-faad82b0fc72 service swift] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:03.764658 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab3df5-8522-4275-adf6-faad82b0fc72 service swift] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:03.764937 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab3df5-8522-4275-adf6-faad82b0fc72 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:03.774203 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccab3df5-8522-4275-adf6-faad82b0fc72 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:03.775870 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccab3df5-8522-4275-adf6-faad82b0fc72 service swift] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:03.785019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1283/2565] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Jun 15 14:52:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:52:13.639454 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=48,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:13.781684 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=74,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:24.062475 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7939cfd5-5446-4172-9152-32ef3e4aae9e None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:24.063991 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-7939cfd5-5446-4172-9152-32ef3e4aae9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:24.145071 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7939cfd5-5446-4172-9152-32ef3e4aae9e None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:24.149659 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c37e35-a091-4631-9100-d8b68691b05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:24.150537 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c37e35-a091-4631-9100-d8b68691b05a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:24.150819 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c37e35-a091-4631-9100-d8b68691b05a admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:24.151122 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c37e35-a091-4631-9100-d8b68691b05a admin admin] PATH_INFO: `/v3/users/bc1a5ba5dbfa404dad6aacc16fa3993c` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:24.152408 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c37e35-a091-4631-9100-d8b68691b05a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc1a5ba5dbfa404dad6aacc16fa3993c)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:24.154343 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c37e35-a091-4631-9100-d8b68691b05a admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:24.216103 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-20c37e35-a091-4631-9100-d8b68691b05a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc1a5ba5dbfa404dad6aacc16fa3993c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:24.216803 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-20c37e35-a091-4631-9100-d8b68691b05a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc1a5ba5dbfa404dad6aacc16fa3993c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:24.233183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-20c37e35-a091-4631-9100-d8b68691b05a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc1a5ba5dbfa404dad6aacc16fa3993c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:24.240664 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-20c37e35-a091-4631-9100-d8b68691b05a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc1a5ba5dbfa404dad6aacc16fa3993c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:24.248068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-20c37e35-a091-4631-9100-d8b68691b05a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc1a5ba5dbfa404dad6aacc16fa3993c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:24.254411 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1283/2566] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:52:24 2022] DELETE /identity/v3/users/bc1a5ba5dbfa404dad6aacc16fa3993c => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:24.258982 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc35846-446b-47e1-9a8f-67bbd5ea987b None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:24.259499 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc35846-446b-47e1-9a8f-67bbd5ea987b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:24.259591 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc35846-446b-47e1-9a8f-67bbd5ea987b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:24.277895 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-5cc35846-446b-47e1-9a8f-67bbd5ea987b None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:24.279152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5cc35846-446b-47e1-9a8f-67bbd5ea987b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:24.339104 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1284/2567] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:52:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:24.349581 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c37e35-a091-4631-9100-d8b68691b05a admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:24.351121 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-20c37e35-a091-4631-9100-d8b68691b05a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:24.413372 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c37e35-a091-4631-9100-d8b68691b05a admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:24.417063 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc566c8d-fc20-45b9-b5d1-8f8e03bde2b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:24.417709 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc566c8d-fc20-45b9-b5d1-8f8e03bde2b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:24.417994 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc566c8d-fc20-45b9-b5d1-8f8e03bde2b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:24.418271 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc566c8d-fc20-45b9-b5d1-8f8e03bde2b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:24.427890 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc566c8d-fc20-45b9-b5d1-8f8e03bde2b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:24.429336 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc566c8d-fc20-45b9-b5d1-8f8e03bde2b1 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:24.439600 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1284/2568] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:24.801692 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc35846-446b-47e1-9a8f-67bbd5ea987b None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:24.804067 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5cc35846-446b-47e1-9a8f-67bbd5ea987b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:24.821250 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc35846-446b-47e1-9a8f-67bbd5ea987b None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:24.840269 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0904bc8b-dff5-45d9-a506-bbda709f94d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:24.840269 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904bc8b-dff5-45d9-a506-bbda709f94d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:24.840269 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904bc8b-dff5-45d9-a506-bbda709f94d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:24.840269 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904bc8b-dff5-45d9-a506-bbda709f94d2 admin admin] PATH_INFO: `/v3/projects/af085f0f0b8945bd9ec5aac95af67955` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:24.840889 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0904bc8b-dff5-45d9-a506-bbda709f94d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af085f0f0b8945bd9ec5aac95af67955)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:24.840889 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0904bc8b-dff5-45d9-a506-bbda709f94d2 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:24.908843 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-0904bc8b-dff5-45d9-a506-bbda709f94d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af085f0f0b8945bd9ec5aac95af67955', 'request_id': 'req-0904bc8b-dff5-45d9-a506-bbda709f94d2'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:24.921820 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-0904bc8b-dff5-45d9-a506-bbda709f94d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af085f0f0b8945bd9ec5aac95af67955', 'request_id': 'req-0904bc8b-dff5-45d9-a506-bbda709f94d2'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:24.928982 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-0904bc8b-dff5-45d9-a506-bbda709f94d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af085f0f0b8945bd9ec5aac95af67955', 'request_id': 'req-0904bc8b-dff5-45d9-a506-bbda709f94d2'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:24.940710 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-0904bc8b-dff5-45d9-a506-bbda709f94d2 admin admin] The token cache is being invalidate because project af085f0f0b8945bd9ec5aac95af67955 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:52:24.940968 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-0904bc8b-dff5-45d9-a506-bbda709f94d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:24.941644 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-0904bc8b-dff5-45d9-a506-bbda709f94d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:24.943275 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1285/2569] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:52:24 2022] DELETE /identity/v3/projects/af085f0f0b8945bd9ec5aac95af67955 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:24.949090 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb9574-fdcb-4284-af87-13c73b2543c4 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:24.949466 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb9574-fdcb-4284-af87-13c73b2543c4 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:24.949809 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb9574-fdcb-4284-af87-13c73b2543c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:24.977590 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-f4bb9574-fdcb-4284-af87-13c73b2543c4 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:24.978833 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f4bb9574-fdcb-4284-af87-13c73b2543c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:24.998247 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:25.049185 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1285/2570] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:52:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:25.058123 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc17a47-123f-42bd-ac6a-606af8c00e95 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:25.058123 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc17a47-123f-42bd-ac6a-606af8c00e95 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:25.058415 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc17a47-123f-42bd-ac6a-606af8c00e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:25.078445 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-6dc17a47-123f-42bd-ac6a-606af8c00e95 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:25.079733 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6dc17a47-123f-42bd-ac6a-606af8c00e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:25.090314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1286/2571] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:52:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:25.095273 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb9574-fdcb-4284-af87-13c73b2543c4 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:25.105742 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb9574-fdcb-4284-af87-13c73b2543c4 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:25.110516 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36093aa-c9e9-4122-9698-3a7b343158a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:25.111232 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36093aa-c9e9-4122-9698-3a7b343158a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:25.111521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36093aa-c9e9-4122-9698-3a7b343158a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:25.111794 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36093aa-c9e9-4122-9698-3a7b343158a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:25.112526 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36093aa-c9e9-4122-9698-3a7b343158a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:52:25.112834 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36093aa-c9e9-4122-9698-3a7b343158a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:25.114482 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36093aa-c9e9-4122-9698-3a7b343158a4 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:25.128246 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1286/2572] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:52:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:52:25.132789 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc17a47-123f-42bd-ac6a-606af8c00e95 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:25.143494 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc17a47-123f-42bd-ac6a-606af8c00e95 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:25.148499 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2fdb63-dee1-4ad2-9323-dd4b9ad5c4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:25.149172 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2fdb63-dee1-4ad2-9323-dd4b9ad5c4f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:25.149459 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2fdb63-dee1-4ad2-9323-dd4b9ad5c4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:25.149736 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2fdb63-dee1-4ad2-9323-dd4b9ad5c4f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:25.150438 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2fdb63-dee1-4ad2-9323-dd4b9ad5c4f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:25.152460 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2fdb63-dee1-4ad2-9323-dd4b9ad5c4f2 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:25.190170 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1287/2573] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:52:25 2022] POST /identity/v3/projects => generated 381 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:52:25.194578 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36093aa-c9e9-4122-9698-3a7b343158a4 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:25.205409 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36093aa-c9e9-4122-9698-3a7b343158a4 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:25.269971 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1507c6d-a6ab-4612-8fd8-7a1d1581f805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:25.270661 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1507c6d-a6ab-4612-8fd8-7a1d1581f805 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:25.270947 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1507c6d-a6ab-4612-8fd8-7a1d1581f805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:25.271257 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1507c6d-a6ab-4612-8fd8-7a1d1581f805 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:25.271944 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1507c6d-a6ab-4612-8fd8-7a1d1581f805 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:25.273922 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1507c6d-a6ab-4612-8fd8-7a1d1581f805 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:25.294792 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1287/2574] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:52:25 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:52:25.300121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2fdb63-dee1-4ad2-9323-dd4b9ad5c4f2 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:25.310682 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2fdb63-dee1-4ad2-9323-dd4b9ad5c4f2 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:25.316201 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36d11f3-8e72-409b-94b8-79e07aa8371a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:25.316872 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d11f3-8e72-409b-94b8-79e07aa8371a admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:25.317160 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d11f3-8e72-409b-94b8-79e07aa8371a admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:25.317444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d11f3-8e72-409b-94b8-79e07aa8371a admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:25.318180 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36d11f3-8e72-409b-94b8-79e07aa8371a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:25.319844 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36d11f3-8e72-409b-94b8-79e07aa8371a admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:25.331955 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1288/2575] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:25.336399 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1507c6d-a6ab-4612-8fd8-7a1d1581f805 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:25.346841 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1507c6d-a6ab-4612-8fd8-7a1d1581f805 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:25.352519 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d01a49-df72-4db6-9566-708523b13349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:25.353188 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d01a49-df72-4db6-9566-708523b13349 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:25.353470 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d01a49-df72-4db6-9566-708523b13349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:25.353747 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d01a49-df72-4db6-9566-708523b13349 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:25.354485 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d01a49-df72-4db6-9566-708523b13349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:25.356136 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d01a49-df72-4db6-9566-708523b13349 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:25.369556 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1288/2576] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:25.373852 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36d11f3-8e72-409b-94b8-79e07aa8371a admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:25.384923 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36d11f3-8e72-409b-94b8-79e07aa8371a admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:25.389800 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a9eca0-5d7f-4b90-b7e2-dedc631e7a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:25.391280 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9eca0-5d7f-4b90-b7e2-dedc631e7a80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:25.391590 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9eca0-5d7f-4b90-b7e2-dedc631e7a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:25.391867 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9eca0-5d7f-4b90-b7e2-dedc631e7a80 admin admin] PATH_INFO: `/v3/projects/28dfd45278d84a798da8e77bf7e6bc9d/users/235cd365a5ab41cda9bffd0cee150617/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:25.407564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a9eca0-5d7f-4b90-b7e2-dedc631e7a80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28dfd45278d84a798da8e77bf7e6bc9d, user_id=235cd365a5ab41cda9bffd0cee150617, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:25.410236 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a9eca0-5d7f-4b90-b7e2-dedc631e7a80 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:25.425475 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1289/2577] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:25 2022] PUT /identity/v3/projects/28dfd45278d84a798da8e77bf7e6bc9d/users/235cd365a5ab41cda9bffd0cee150617/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:25.430746 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce370e-2bf7-44bc-be55-84025132725b None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:25.431058 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce370e-2bf7-44bc-be55-84025132725b None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:25.431344 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce370e-2bf7-44bc-be55-84025132725b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:25.450311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-e9ce370e-2bf7-44bc-be55-84025132725b None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:25.451569 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e9ce370e-2bf7-44bc-be55-84025132725b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:25.513872 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1289/2578] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:52:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:25.523681 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a9eca0-5d7f-4b90-b7e2-dedc631e7a80 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:25.525384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-75a9eca0-5d7f-4b90-b7e2-dedc631e7a80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:25.548436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:25.583978 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a9eca0-5d7f-4b90-b7e2-dedc631e7a80 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:25.587566 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b11175-6588-4cae-8b2e-73d84d3c21e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:25.588216 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b11175-6588-4cae-8b2e-73d84d3c21e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:25.588498 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b11175-6588-4cae-8b2e-73d84d3c21e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:25.588776 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b11175-6588-4cae-8b2e-73d84d3c21e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:25.597783 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b11175-6588-4cae-8b2e-73d84d3c21e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:25.599256 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b11175-6588-4cae-8b2e-73d84d3c21e5 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:25.609897 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1290/2579] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:26.321500 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfc813-bb08-410c-8ebc-4ff43e0fefcc None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:26.322082 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfc813-bb08-410c-8ebc-4ff43e0fefcc None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:26.322082 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfc813-bb08-410c-8ebc-4ff43e0fefcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:26.351879 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-05bfc813-bb08-410c-8ebc-4ff43e0fefcc None None] MFA Rules not processed for user `235cd365a5ab41cda9bffd0cee150617`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:26.353055 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-05bfc813-bb08-410c-8ebc-4ff43e0fefcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:26.370635 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:26.422555 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1290/2580] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:52:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:26.433241 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b11175-6588-4cae-8b2e-73d84d3c21e5 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:26.434745 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-60b11175-6588-4cae-8b2e-73d84d3c21e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:26.497857 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b11175-6588-4cae-8b2e-73d84d3c21e5 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:26.501979 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d335a9f-8550-4414-adfe-d62c6db779f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:26.502662 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d335a9f-8550-4414-adfe-d62c6db779f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:26.502957 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d335a9f-8550-4414-adfe-d62c6db779f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:26.503272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d335a9f-8550-4414-adfe-d62c6db779f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:26.513073 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d335a9f-8550-4414-adfe-d62c6db779f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:26.514617 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d335a9f-8550-4414-adfe-d62c6db779f0 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:26.525466 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1291/2581] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:27.186289 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a5796-ef7c-4fc0-965c-1e5f93ffa9b5 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:27.186822 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a5796-ef7c-4fc0-965c-1e5f93ffa9b5 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:27.187971 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a5796-ef7c-4fc0-965c-1e5f93ffa9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:27.210337 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-ed0a5796-ef7c-4fc0-965c-1e5f93ffa9b5 None None] MFA Rules not processed for user `bf393488726e49338e70017f5cb7c4be`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:27.211832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ed0a5796-ef7c-4fc0-965c-1e5f93ffa9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:27.274643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1291/2582] 10.222.0.45 () {58 vars in 967 bytes} [Wed Jun 15 14:52:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:52:27.299405 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d335a9f-8550-4414-adfe-d62c6db779f0 service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:27.300834 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-3d335a9f-8550-4414-adfe-d62c6db779f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:27.373008 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d335a9f-8550-4414-adfe-d62c6db779f0 service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:27.378215 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d381bf2-d109-4740-9655-62b8d23629fb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '641a81b3140f47119b5c2323a97a6681', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:27.378883 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d381bf2-d109-4740-9655-62b8d23629fb service swift] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:27.379198 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d381bf2-d109-4740-9655-62b8d23629fb service swift] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:27.379481 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d381bf2-d109-4740-9655-62b8d23629fb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:27.387936 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d381bf2-d109-4740-9655-62b8d23629fb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:27.389533 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d381bf2-d109-4740-9655-62b8d23629fb service swift] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:27.397728 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1292/2583] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Jun 15 14:52:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:52:27.878898 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580dfd05-231a-455c-8c29-f277de4a5ebe None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:27.879450 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580dfd05-231a-455c-8c29-f277de4a5ebe None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:27.879572 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580dfd05-231a-455c-8c29-f277de4a5ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:27.897946 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-580dfd05-231a-455c-8c29-f277de4a5ebe None None] MFA Rules not processed for user `235cd365a5ab41cda9bffd0cee150617`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:27.899197 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-580dfd05-231a-455c-8c29-f277de4a5ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:27.907933 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1292/2584] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:52:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:37.394359 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=75 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:37.892380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=47 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:46.979992 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d381bf2-d109-4740-9655-62b8d23629fb service swift] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:46.981319 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5d381bf2-d109-4740-9655-62b8d23629fb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:46.993978 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d381bf2-d109-4740-9655-62b8d23629fb service swift] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:46.997504 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68660a16-7195-410f-9415-933983828da9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:46.998132 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68660a16-7195-410f-9415-933983828da9 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:46.998418 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68660a16-7195-410f-9415-933983828da9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:46.998719 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68660a16-7195-410f-9415-933983828da9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:47.000490 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-68660a16-7195-410f-9415-933983828da9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:47.011989 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68660a16-7195-410f-9415-933983828da9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:47.013400 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68660a16-7195-410f-9415-933983828da9 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:47.022928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1293/2585] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:50.258442 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580dfd05-231a-455c-8c29-f277de4a5ebe None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:50.267198 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-580dfd05-231a-455c-8c29-f277de4a5ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:50.282195 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580dfd05-231a-455c-8c29-f277de4a5ebe None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:50.287802 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540e07f6-be7e-4a64-bc63-fe05736176f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:50.289068 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e07f6-be7e-4a64-bc63-fe05736176f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:50.289359 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e07f6-be7e-4a64-bc63-fe05736176f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:50.289617 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e07f6-be7e-4a64-bc63-fe05736176f5 admin admin] PATH_INFO: `/v3/users/1efb0951ba1144d58698e02e0e5e9416` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:50.304493 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540e07f6-be7e-4a64-bc63-fe05736176f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1efb0951ba1144d58698e02e0e5e9416)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:50.306518 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540e07f6-be7e-4a64-bc63-fe05736176f5 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:50.356647 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-540e07f6-be7e-4a64-bc63-fe05736176f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1efb0951ba1144d58698e02e0e5e9416'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:50.357291 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-540e07f6-be7e-4a64-bc63-fe05736176f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1efb0951ba1144d58698e02e0e5e9416'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:50.370192 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-540e07f6-be7e-4a64-bc63-fe05736176f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1efb0951ba1144d58698e02e0e5e9416'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:50.377302 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-540e07f6-be7e-4a64-bc63-fe05736176f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1efb0951ba1144d58698e02e0e5e9416'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:50.385787 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-540e07f6-be7e-4a64-bc63-fe05736176f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1efb0951ba1144d58698e02e0e5e9416'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:50.392216 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1293/2586] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:52:50 2022] DELETE /identity/v3/users/1efb0951ba1144d58698e02e0e5e9416 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:50.569990 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68660a16-7195-410f-9415-933983828da9 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:50.582234 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68660a16-7195-410f-9415-933983828da9 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:50.640900 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80b5e6-87b6-48c5-9d7c-7c684bb64be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:50.641755 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80b5e6-87b6-48c5-9d7c-7c684bb64be7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:50.642033 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80b5e6-87b6-48c5-9d7c-7c684bb64be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:50.642303 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80b5e6-87b6-48c5-9d7c-7c684bb64be7 admin admin] PATH_INFO: `/v3/projects/6d3ce7cecd2f4983b8442f70ed2afdb1` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:50.653366 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c80b5e6-87b6-48c5-9d7c-7c684bb64be7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d3ce7cecd2f4983b8442f70ed2afdb1)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:50.656361 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c80b5e6-87b6-48c5-9d7c-7c684bb64be7 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:50.706624 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-1c80b5e6-87b6-48c5-9d7c-7c684bb64be7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d3ce7cecd2f4983b8442f70ed2afdb1', 'request_id': 'req-1c80b5e6-87b6-48c5-9d7c-7c684bb64be7'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:50.714005 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-1c80b5e6-87b6-48c5-9d7c-7c684bb64be7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d3ce7cecd2f4983b8442f70ed2afdb1', 'request_id': 'req-1c80b5e6-87b6-48c5-9d7c-7c684bb64be7'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:50.721231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-1c80b5e6-87b6-48c5-9d7c-7c684bb64be7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d3ce7cecd2f4983b8442f70ed2afdb1', 'request_id': 'req-1c80b5e6-87b6-48c5-9d7c-7c684bb64be7'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:50.731981 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-1c80b5e6-87b6-48c5-9d7c-7c684bb64be7 admin admin] The token cache is being invalidate because project 6d3ce7cecd2f4983b8442f70ed2afdb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:52:50.732267 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-1c80b5e6-87b6-48c5-9d7c-7c684bb64be7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:50.732924 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-1c80b5e6-87b6-48c5-9d7c-7c684bb64be7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:50.734500 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1294/2587] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:52:50 2022] DELETE /identity/v3/projects/6d3ce7cecd2f4983b8442f70ed2afdb1 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:50.739349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec7f1f-8c7a-462a-ac39-9cd7a9d15a71 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:50.739645 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec7f1f-8c7a-462a-ac39-9cd7a9d15a71 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:50.739930 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec7f1f-8c7a-462a-ac39-9cd7a9d15a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:50.760812 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-ccec7f1f-8c7a-462a-ac39-9cd7a9d15a71 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:50.762024 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ccec7f1f-8c7a-462a-ac39-9cd7a9d15a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:50.823236 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1294/2588] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:52:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:50.831759 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b126c1-e7a0-415e-bd68-14f0755cf11e None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:50.832048 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b126c1-e7a0-415e-bd68-14f0755cf11e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:50.832327 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b126c1-e7a0-415e-bd68-14f0755cf11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:50.851384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-e8b126c1-e7a0-415e-bd68-14f0755cf11e None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:50.852150 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e8b126c1-e7a0-415e-bd68-14f0755cf11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:50.861336 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1295/2589] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:52:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:50.865724 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccec7f1f-8c7a-462a-ac39-9cd7a9d15a71 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:50.875825 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccec7f1f-8c7a-462a-ac39-9cd7a9d15a71 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:50.881323 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a6dcd-d2c0-4caa-a3cb-5504089f3a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:50.881990 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a6dcd-d2c0-4caa-a3cb-5504089f3a55 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:50.882275 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a6dcd-d2c0-4caa-a3cb-5504089f3a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:50.882554 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a6dcd-d2c0-4caa-a3cb-5504089f3a55 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:50.883321 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62a6dcd-d2c0-4caa-a3cb-5504089f3a55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:52:50.883635 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62a6dcd-d2c0-4caa-a3cb-5504089f3a55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:50.885271 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62a6dcd-d2c0-4caa-a3cb-5504089f3a55 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:50.898638 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1295/2590] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:52:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:52:50.903182 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b126c1-e7a0-415e-bd68-14f0755cf11e None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:50.913086 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b126c1-e7a0-415e-bd68-14f0755cf11e None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:50.917779 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095ba46-bf27-4dcb-9194-25431522f99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:50.918433 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095ba46-bf27-4dcb-9194-25431522f99f admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:50.918713 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095ba46-bf27-4dcb-9194-25431522f99f admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:50.919149 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095ba46-bf27-4dcb-9194-25431522f99f admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:50.919840 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a095ba46-bf27-4dcb-9194-25431522f99f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:50.921759 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a095ba46-bf27-4dcb-9194-25431522f99f admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:50.956003 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1296/2591] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:52:50 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:52:50.960398 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a6dcd-d2c0-4caa-a3cb-5504089f3a55 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:50.970285 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a6dcd-d2c0-4caa-a3cb-5504089f3a55 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:51.016559 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:51.029684 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82734540-752c-40c3-bc4c-db0578ce82e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:51.030351 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82734540-752c-40c3-bc4c-db0578ce82e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:51.030635 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82734540-752c-40c3-bc4c-db0578ce82e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:51.030908 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82734540-752c-40c3-bc4c-db0578ce82e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:51.031628 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82734540-752c-40c3-bc4c-db0578ce82e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:51.033559 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82734540-752c-40c3-bc4c-db0578ce82e5 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:51.059404 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1296/2592] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:52:50 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:52:51.063587 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095ba46-bf27-4dcb-9194-25431522f99f admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:51.074484 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095ba46-bf27-4dcb-9194-25431522f99f admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:51.079370 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7499d-0406-4998-87c0-39f00f578a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:51.080034 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7499d-0406-4998-87c0-39f00f578a49 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:51.080349 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7499d-0406-4998-87c0-39f00f578a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:51.080629 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7499d-0406-4998-87c0-39f00f578a49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:51.081392 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe7499d-0406-4998-87c0-39f00f578a49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:51.083098 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe7499d-0406-4998-87c0-39f00f578a49 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:51.093189 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1297/2593] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:51.097141 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82734540-752c-40c3-bc4c-db0578ce82e5 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:51.107039 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82734540-752c-40c3-bc4c-db0578ce82e5 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:51.111581 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be813e-ac98-4868-8975-a8a2e196f357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:51.112757 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be813e-ac98-4868-8975-a8a2e196f357 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:51.113054 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be813e-ac98-4868-8975-a8a2e196f357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:51.113314 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be813e-ac98-4868-8975-a8a2e196f357 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:51.114027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be813e-ac98-4868-8975-a8a2e196f357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:51.115630 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be813e-ac98-4868-8975-a8a2e196f357 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:51.126027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1297/2594] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:52:51.130576 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7499d-0406-4998-87c0-39f00f578a49 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:51.140946 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7499d-0406-4998-87c0-39f00f578a49 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:51.146353 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c98163-9e19-4900-be38-1205d1a1cf1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:51.147319 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c98163-9e19-4900-be38-1205d1a1cf1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:51.147614 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c98163-9e19-4900-be38-1205d1a1cf1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:51.147894 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c98163-9e19-4900-be38-1205d1a1cf1e admin admin] PATH_INFO: `/v3/projects/8247826bc0b943e7b511c2b2000467d5/users/880a5d9dab664048b5fc748825a78f93/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:51.163559 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c98163-9e19-4900-be38-1205d1a1cf1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8247826bc0b943e7b511c2b2000467d5, user_id=880a5d9dab664048b5fc748825a78f93, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:51.166273 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c98163-9e19-4900-be38-1205d1a1cf1e admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:51.180455 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1298/2595] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:51 2022] PUT /identity/v3/projects/8247826bc0b943e7b511c2b2000467d5/users/880a5d9dab664048b5fc748825a78f93/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:51.192120 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16703b7d-5ecb-4120-953d-7b4b7d500dad None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:51.192443 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16703b7d-5ecb-4120-953d-7b4b7d500dad None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:51.192727 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16703b7d-5ecb-4120-953d-7b4b7d500dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:51.220239 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-16703b7d-5ecb-4120-953d-7b4b7d500dad None None] MFA Rules not processed for user `880a5d9dab664048b5fc748825a78f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:51.221440 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-16703b7d-5ecb-4120-953d-7b4b7d500dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:51.294232 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1298/2596] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:52:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:51.303865 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c98163-9e19-4900-be38-1205d1a1cf1e admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:51.309457 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:51.370920 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c98163-9e19-4900-be38-1205d1a1cf1e admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:51.375699 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e8bad7-0ed2-4832-aa98-7e03ed3e5f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:51.376355 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8bad7-0ed2-4832-aa98-7e03ed3e5f01 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:51.376641 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8bad7-0ed2-4832-aa98-7e03ed3e5f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:51.376920 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8bad7-0ed2-4832-aa98-7e03ed3e5f01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:51.377645 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e8bad7-0ed2-4832-aa98-7e03ed3e5f01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:51.379579 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e8bad7-0ed2-4832-aa98-7e03ed3e5f01 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:51.403864 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1299/2597] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:52:51 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:52:51.408008 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16703b7d-5ecb-4120-953d-7b4b7d500dad None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:51.419524 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16703b7d-5ecb-4120-953d-7b4b7d500dad None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:51.478869 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d5cad-b8b4-471c-a4ce-59854cc4d949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:51.479489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d5cad-b8b4-471c-a4ce-59854cc4d949 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:51.479780 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d5cad-b8b4-471c-a4ce-59854cc4d949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:51.480064 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d5cad-b8b4-471c-a4ce-59854cc4d949 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:51.480762 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591d5cad-b8b4-471c-a4ce-59854cc4d949 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:51.482767 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591d5cad-b8b4-471c-a4ce-59854cc4d949 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:51.503583 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1299/2598] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:52:51 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:52:51.507419 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e8bad7-0ed2-4832-aa98-7e03ed3e5f01 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:51.518723 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e8bad7-0ed2-4832-aa98-7e03ed3e5f01 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:51.523447 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a0b6c8-4318-4a6c-8a6a-b8f227c1f79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:51.524110 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0b6c8-4318-4a6c-8a6a-b8f227c1f79d admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:51.524394 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0b6c8-4318-4a6c-8a6a-b8f227c1f79d admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:51.524700 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0b6c8-4318-4a6c-8a6a-b8f227c1f79d admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:51.525432 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a0b6c8-4318-4a6c-8a6a-b8f227c1f79d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:51.527063 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a0b6c8-4318-4a6c-8a6a-b8f227c1f79d admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:51.538381 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1300/2599] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:51.542603 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d5cad-b8b4-471c-a4ce-59854cc4d949 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:51.553419 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d5cad-b8b4-471c-a4ce-59854cc4d949 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:51.558253 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb47d0-e4fc-4da5-b48d-1c4e8743f315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:51.559206 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb47d0-e4fc-4da5-b48d-1c4e8743f315 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:51.559495 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb47d0-e4fc-4da5-b48d-1c4e8743f315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:51.559780 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb47d0-e4fc-4da5-b48d-1c4e8743f315 admin admin] PATH_INFO: `/v3/projects/02f7a8fb2c1841089e08908320c179e3/users/75491ba085de4df99d96f2a5abddddb1/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:51.574637 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befb47d0-e4fc-4da5-b48d-1c4e8743f315 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02f7a8fb2c1841089e08908320c179e3, user_id=75491ba085de4df99d96f2a5abddddb1, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:51.577325 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befb47d0-e4fc-4da5-b48d-1c4e8743f315 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:51.590728 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1300/2600] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:51 2022] PUT /identity/v3/projects/02f7a8fb2c1841089e08908320c179e3/users/75491ba085de4df99d96f2a5abddddb1/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:51.594672 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a0b6c8-4318-4a6c-8a6a-b8f227c1f79d admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:51.606046 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a0b6c8-4318-4a6c-8a6a-b8f227c1f79d admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:51.664023 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c577b-7b5b-4ca9-a40c-5233f9bd35f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:51.664718 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c577b-7b5b-4ca9-a40c-5233f9bd35f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:51.665000 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c577b-7b5b-4ca9-a40c-5233f9bd35f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:51.665273 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c577b-7b5b-4ca9-a40c-5233f9bd35f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:51.665991 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c577b-7b5b-4ca9-a40c-5233f9bd35f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:51.667632 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c577b-7b5b-4ca9-a40c-5233f9bd35f2 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:51.677346 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1301/2601] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:51 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:51.681508 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb47d0-e4fc-4da5-b48d-1c4e8743f315 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:51.692278 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb47d0-e4fc-4da5-b48d-1c4e8743f315 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:51.697863 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d4e47-1172-44d1-9472-d556ae34f111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:51.699094 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d4e47-1172-44d1-9472-d556ae34f111 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:51.699381 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d4e47-1172-44d1-9472-d556ae34f111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:51.699667 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d4e47-1172-44d1-9472-d556ae34f111 admin admin] PATH_INFO: `/v3/projects/02f7a8fb2c1841089e08908320c179e3/users/75491ba085de4df99d96f2a5abddddb1/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:51.702040 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44d4e47-1172-44d1-9472-d556ae34f111 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02f7a8fb2c1841089e08908320c179e3, user_id=75491ba085de4df99d96f2a5abddddb1, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:51.704779 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44d4e47-1172-44d1-9472-d556ae34f111 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:51.716642 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1301/2602] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:51 2022] PUT /identity/v3/projects/02f7a8fb2c1841089e08908320c179e3/users/75491ba085de4df99d96f2a5abddddb1/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:51.727402 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bfb79-56b8-43b9-9594-f0a99461b813 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:51.727690 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bfb79-56b8-43b9-9594-f0a99461b813 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:51.727964 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bfb79-56b8-43b9-9594-f0a99461b813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:51.753596 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-dd7bfb79-56b8-43b9-9594-f0a99461b813 None None] MFA Rules not processed for user `75491ba085de4df99d96f2a5abddddb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:51.754636 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-dd7bfb79-56b8-43b9-9594-f0a99461b813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:51.824590 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:51.828818 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1302/2603] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:52:51 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:51.840127 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d4e47-1172-44d1-9472-d556ae34f111 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:51.841876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e44d4e47-1172-44d1-9472-d556ae34f111 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:51.853626 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:51.901474 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d4e47-1172-44d1-9472-d556ae34f111 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:51.905478 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390f921-4ac7-4635-9b03-251a7b958a09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:51.906125 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390f921-4ac7-4635-9b03-251a7b958a09 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:51.906412 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390f921-4ac7-4635-9b03-251a7b958a09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:51.906692 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390f921-4ac7-4635-9b03-251a7b958a09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:51.965923 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8390f921-4ac7-4635-9b03-251a7b958a09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:51.967555 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8390f921-4ac7-4635-9b03-251a7b958a09 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:51.977365 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1302/2604] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:00.689539 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7bfb79-56b8-43b9-9594-f0a99461b813 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:00.690937 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-dd7bfb79-56b8-43b9-9594-f0a99461b813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:00.704239 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7bfb79-56b8-43b9-9594-f0a99461b813 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:00.707936 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d3f46-9689-435d-9080-6e1c825aa565 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:00.708589 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d3f46-9689-435d-9080-6e1c825aa565 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:00.708875 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d3f46-9689-435d-9080-6e1c825aa565 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:00.709154 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d3f46-9689-435d-9080-6e1c825aa565 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:00.718640 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d3f46-9689-435d-9080-6e1c825aa565 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:00.720108 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d3f46-9689-435d-9080-6e1c825aa565 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:00.731808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1303/2605] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:00 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:01.973409 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=44 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:10.725516 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=6 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:19.667499 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390f921-4ac7-4635-9b03-251a7b958a09 service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:19.676142 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390f921-4ac7-4635-9b03-251a7b958a09 service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:19.680258 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8461e143-9bd8-4f68-a5d9-757236614e2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:19.680892 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461e143-9bd8-4f68-a5d9-757236614e2d service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:19.681176 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461e143-9bd8-4f68-a5d9-757236614e2d service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:19.681455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461e143-9bd8-4f68-a5d9-757236614e2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:19.683237 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-8461e143-9bd8-4f68-a5d9-757236614e2d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:19.754067 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8461e143-9bd8-4f68-a5d9-757236614e2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:19.755640 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8461e143-9bd8-4f68-a5d9-757236614e2d service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:19.766519 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1303/2606] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:29.761568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=34 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:39.859628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d3f46-9689-435d-9080-6e1c825aa565 service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:39.921560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d3f46-9689-435d-9080-6e1c825aa565 service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:39.926128 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01dfe4-0cb5-4dca-8993-cd8361bddab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:39.927020 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01dfe4-0cb5-4dca-8993-cd8361bddab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:39.927313 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01dfe4-0cb5-4dca-8993-cd8361bddab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:39.927592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01dfe4-0cb5-4dca-8993-cd8361bddab4 admin admin] PATH_INFO: `/v3/users/880a5d9dab664048b5fc748825a78f93` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:39.928799 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d01dfe4-0cb5-4dca-8993-cd8361bddab4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=880a5d9dab664048b5fc748825a78f93)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:39.931403 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d01dfe4-0cb5-4dca-8993-cd8361bddab4 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:39.984369 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-2d01dfe4-0cb5-4dca-8993-cd8361bddab4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '880a5d9dab664048b5fc748825a78f93'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:39.984982 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-2d01dfe4-0cb5-4dca-8993-cd8361bddab4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '880a5d9dab664048b5fc748825a78f93'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:39.997425 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-2d01dfe4-0cb5-4dca-8993-cd8361bddab4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '880a5d9dab664048b5fc748825a78f93'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:39.998061 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-2d01dfe4-0cb5-4dca-8993-cd8361bddab4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '880a5d9dab664048b5fc748825a78f93'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:40.006130 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-2d01dfe4-0cb5-4dca-8993-cd8361bddab4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '880a5d9dab664048b5fc748825a78f93'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:40.018556 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1304/2607] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:53:39 2022] DELETE /identity/v3/users/880a5d9dab664048b5fc748825a78f93 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:40.023102 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5183f2c-c3df-4860-9312-b2a7dc72ebd8 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:40.023616 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5183f2c-c3df-4860-9312-b2a7dc72ebd8 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:40.023719 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5183f2c-c3df-4860-9312-b2a7dc72ebd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:40.041060 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-d5183f2c-c3df-4860-9312-b2a7dc72ebd8 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:40.042266 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d5183f2c-c3df-4860-9312-b2a7dc72ebd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:40.100408 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1304/2608] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:53:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:40.111160 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01dfe4-0cb5-4dca-8993-cd8361bddab4 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:40.112546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-2d01dfe4-0cb5-4dca-8993-cd8361bddab4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:40.170533 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01dfe4-0cb5-4dca-8993-cd8361bddab4 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:40.176503 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440527a7-4fcb-4979-94ac-bf8bf688ce8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:40.177160 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440527a7-4fcb-4979-94ac-bf8bf688ce8b service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:40.177442 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440527a7-4fcb-4979-94ac-bf8bf688ce8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:40.177745 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440527a7-4fcb-4979-94ac-bf8bf688ce8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:40.188140 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440527a7-4fcb-4979-94ac-bf8bf688ce8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:40.189712 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440527a7-4fcb-4979-94ac-bf8bf688ce8b service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:40.199081 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1305/2609] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:40.467792 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5183f2c-c3df-4860-9312-b2a7dc72ebd8 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:40.469238 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d5183f2c-c3df-4860-9312-b2a7dc72ebd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:40.481337 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5183f2c-c3df-4860-9312-b2a7dc72ebd8 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:40.486091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c6f47-b169-4156-a203-d66a52929b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:40.486952 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c6f47-b169-4156-a203-d66a52929b94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:40.487281 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c6f47-b169-4156-a203-d66a52929b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:40.487562 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c6f47-b169-4156-a203-d66a52929b94 admin admin] PATH_INFO: `/v3/projects/8247826bc0b943e7b511c2b2000467d5` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:40.488774 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959c6f47-b169-4156-a203-d66a52929b94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8247826bc0b943e7b511c2b2000467d5)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:40.490676 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959c6f47-b169-4156-a203-d66a52929b94 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:40.537581 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-959c6f47-b169-4156-a203-d66a52929b94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8247826bc0b943e7b511c2b2000467d5', 'request_id': 'req-959c6f47-b169-4156-a203-d66a52929b94'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:40.545183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-959c6f47-b169-4156-a203-d66a52929b94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8247826bc0b943e7b511c2b2000467d5', 'request_id': 'req-959c6f47-b169-4156-a203-d66a52929b94'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:40.551516 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-959c6f47-b169-4156-a203-d66a52929b94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8247826bc0b943e7b511c2b2000467d5', 'request_id': 'req-959c6f47-b169-4156-a203-d66a52929b94'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:40.562013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-959c6f47-b169-4156-a203-d66a52929b94 admin admin] The token cache is being invalidate because project 8247826bc0b943e7b511c2b2000467d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:53:40.562300 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-959c6f47-b169-4156-a203-d66a52929b94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:40.562930 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-959c6f47-b169-4156-a203-d66a52929b94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:40.564499 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1305/2610] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:53:40 2022] DELETE /identity/v3/projects/8247826bc0b943e7b511c2b2000467d5 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:40.568613 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440527a7-4fcb-4979-94ac-bf8bf688ce8b service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:40.578817 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440527a7-4fcb-4979-94ac-bf8bf688ce8b service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:40.597557 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:40.638347 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c4358-d8d3-49bd-be46-d6347c5f0c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:40.639254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c4358-d8d3-49bd-be46-d6347c5f0c26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:40.639539 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c4358-d8d3-49bd-be46-d6347c5f0c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:40.639812 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c4358-d8d3-49bd-be46-d6347c5f0c26 admin admin] PATH_INFO: `/v3/users/75491ba085de4df99d96f2a5abddddb1` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:40.641009 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c4358-d8d3-49bd-be46-d6347c5f0c26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75491ba085de4df99d96f2a5abddddb1)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:40.642942 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c4358-d8d3-49bd-be46-d6347c5f0c26 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:40.688519 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-640c4358-d8d3-49bd-be46-d6347c5f0c26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75491ba085de4df99d96f2a5abddddb1'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:40.689176 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-640c4358-d8d3-49bd-be46-d6347c5f0c26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75491ba085de4df99d96f2a5abddddb1'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:40.701678 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-640c4358-d8d3-49bd-be46-d6347c5f0c26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75491ba085de4df99d96f2a5abddddb1'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:40.702337 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-640c4358-d8d3-49bd-be46-d6347c5f0c26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75491ba085de4df99d96f2a5abddddb1'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:40.709915 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-640c4358-d8d3-49bd-be46-d6347c5f0c26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75491ba085de4df99d96f2a5abddddb1'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:40.722266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1306/2611] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:53:40 2022] DELETE /identity/v3/users/75491ba085de4df99d96f2a5abddddb1 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:40.990607 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c6f47-b169-4156-a203-d66a52929b94 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:40.992907 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-959c6f47-b169-4156-a203-d66a52929b94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:41.007289 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c6f47-b169-4156-a203-d66a52929b94 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:41.065682 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d235af0d-094f-4be5-b22d-8aa8bd9d7dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:41.066463 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235af0d-094f-4be5-b22d-8aa8bd9d7dab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:41.066749 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235af0d-094f-4be5-b22d-8aa8bd9d7dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:41.067049 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235af0d-094f-4be5-b22d-8aa8bd9d7dab admin admin] PATH_INFO: `/v3/projects/02f7a8fb2c1841089e08908320c179e3` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:41.068205 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d235af0d-094f-4be5-b22d-8aa8bd9d7dab admin admin] RBAC: Authorizing `identity:delete_project(project_id=02f7a8fb2c1841089e08908320c179e3)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:41.070091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d235af0d-094f-4be5-b22d-8aa8bd9d7dab admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:41.103683 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:41.119592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-d235af0d-094f-4be5-b22d-8aa8bd9d7dab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02f7a8fb2c1841089e08908320c179e3', 'request_id': 'req-d235af0d-094f-4be5-b22d-8aa8bd9d7dab'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:41.126533 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-d235af0d-094f-4be5-b22d-8aa8bd9d7dab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02f7a8fb2c1841089e08908320c179e3', 'request_id': 'req-d235af0d-094f-4be5-b22d-8aa8bd9d7dab'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:41.132715 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-d235af0d-094f-4be5-b22d-8aa8bd9d7dab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02f7a8fb2c1841089e08908320c179e3', 'request_id': 'req-d235af0d-094f-4be5-b22d-8aa8bd9d7dab'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:41.143177 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-d235af0d-094f-4be5-b22d-8aa8bd9d7dab admin admin] The token cache is being invalidate because project 02f7a8fb2c1841089e08908320c179e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:53:41.143468 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-d235af0d-094f-4be5-b22d-8aa8bd9d7dab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:41.144102 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-d235af0d-094f-4be5-b22d-8aa8bd9d7dab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:41.145632 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1306/2612] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:53:40 2022] DELETE /identity/v3/projects/02f7a8fb2c1841089e08908320c179e3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:41.151176 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03170c-1f21-4e84-88a3-9445fcab18af None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:41.151649 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03170c-1f21-4e84-88a3-9445fcab18af None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:41.151753 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03170c-1f21-4e84-88a3-9445fcab18af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:41.171176 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-bd03170c-1f21-4e84-88a3-9445fcab18af None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:41.172408 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-bd03170c-1f21-4e84-88a3-9445fcab18af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:41.233229 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1307/2613] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:53:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:41.242033 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00ba3d-4f88-48cf-a078-9d0067130e50 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:41.242355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00ba3d-4f88-48cf-a078-9d0067130e50 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:41.242633 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00ba3d-4f88-48cf-a078-9d0067130e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:41.260634 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-4c00ba3d-4f88-48cf-a078-9d0067130e50 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:41.261823 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-4c00ba3d-4f88-48cf-a078-9d0067130e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:41.269813 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1307/2614] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:53:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:41.274699 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd03170c-1f21-4e84-88a3-9445fcab18af None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:41.284484 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd03170c-1f21-4e84-88a3-9445fcab18af None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:41.289869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39fa12-b83f-4fa9-879e-1156c400c70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:41.290523 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39fa12-b83f-4fa9-879e-1156c400c70d admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:41.290826 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39fa12-b83f-4fa9-879e-1156c400c70d admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:41.291155 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39fa12-b83f-4fa9-879e-1156c400c70d admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:41.291890 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc39fa12-b83f-4fa9-879e-1156c400c70d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:53:41.292198 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc39fa12-b83f-4fa9-879e-1156c400c70d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:41.293803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc39fa12-b83f-4fa9-879e-1156c400c70d admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:41.305081 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1308/2615] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:53:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:53:41.309440 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00ba3d-4f88-48cf-a078-9d0067130e50 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:41.318589 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00ba3d-4f88-48cf-a078-9d0067130e50 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:41.323132 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8c235-9312-4d87-8240-9890e5b2fcde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:41.323771 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8c235-9312-4d87-8240-9890e5b2fcde admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:41.324053 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8c235-9312-4d87-8240-9890e5b2fcde admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:41.324325 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8c235-9312-4d87-8240-9890e5b2fcde admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:41.324997 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c8c235-9312-4d87-8240-9890e5b2fcde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:41.326867 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c8c235-9312-4d87-8240-9890e5b2fcde admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:41.355775 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1308/2616] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:53:41 2022] POST /identity/v3/projects => generated 393 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:53:41.357048 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39fa12-b83f-4fa9-879e-1156c400c70d admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:41.367850 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39fa12-b83f-4fa9-879e-1156c400c70d admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:41.393387 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:41.425939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9610e6a4-adcc-497e-97d9-d086e91c4c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:41.426606 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9610e6a4-adcc-497e-97d9-d086e91c4c08 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:41.426887 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9610e6a4-adcc-497e-97d9-d086e91c4c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:41.427190 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9610e6a4-adcc-497e-97d9-d086e91c4c08 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:41.427867 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9610e6a4-adcc-497e-97d9-d086e91c4c08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:41.429876 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9610e6a4-adcc-497e-97d9-d086e91c4c08 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:41.449907 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1309/2617] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:53:41 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:53:41.453990 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8c235-9312-4d87-8240-9890e5b2fcde admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:41.464841 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8c235-9312-4d87-8240-9890e5b2fcde admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:41.470114 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1d622-d702-4a18-aea8-4c94b9e0ea08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:41.470772 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1d622-d702-4a18-aea8-4c94b9e0ea08 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:41.471086 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1d622-d702-4a18-aea8-4c94b9e0ea08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:41.471395 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1d622-d702-4a18-aea8-4c94b9e0ea08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:41.472127 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba1d622-d702-4a18-aea8-4c94b9e0ea08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:41.473722 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba1d622-d702-4a18-aea8-4c94b9e0ea08 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:41.484037 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1309/2618] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:53:41.488195 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9610e6a4-adcc-497e-97d9-d086e91c4c08 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:41.497946 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9610e6a4-adcc-497e-97d9-d086e91c4c08 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:41.502635 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da9680f-81ac-431e-b7bc-43f1b19cc718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:41.503326 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da9680f-81ac-431e-b7bc-43f1b19cc718 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:41.503722 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da9680f-81ac-431e-b7bc-43f1b19cc718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:41.504021 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da9680f-81ac-431e-b7bc-43f1b19cc718 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:41.504747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da9680f-81ac-431e-b7bc-43f1b19cc718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:41.506346 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da9680f-81ac-431e-b7bc-43f1b19cc718 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:41.516792 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1310/2619] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:53:41.520824 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1d622-d702-4a18-aea8-4c94b9e0ea08 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:41.531413 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1d622-d702-4a18-aea8-4c94b9e0ea08 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:41.535994 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537190a6-bf9c-48cd-9713-77d5e552706d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:41.536869 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537190a6-bf9c-48cd-9713-77d5e552706d admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:41.537152 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537190a6-bf9c-48cd-9713-77d5e552706d admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:41.537425 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537190a6-bf9c-48cd-9713-77d5e552706d admin admin] PATH_INFO: `/v3/projects/339dfd7ff18d454ea37a9c4b4f41885c/users/9bcff29f0f0d444ebae077deb2ff04c4/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:41.552038 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537190a6-bf9c-48cd-9713-77d5e552706d admin admin] RBAC: Authorizing `identity:create_grant(project_id=339dfd7ff18d454ea37a9c4b4f41885c, user_id=9bcff29f0f0d444ebae077deb2ff04c4, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:41.554673 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537190a6-bf9c-48cd-9713-77d5e552706d admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:41.567522 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1310/2620] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:53:41 2022] PUT /identity/v3/projects/339dfd7ff18d454ea37a9c4b4f41885c/users/9bcff29f0f0d444ebae077deb2ff04c4/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:41.578263 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f0933-4866-4b03-a66d-27646f196b6c None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:41.578492 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f0933-4866-4b03-a66d-27646f196b6c None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:41.578786 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f0933-4866-4b03-a66d-27646f196b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:41.605193 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-177f0933-4866-4b03-a66d-27646f196b6c None None] MFA Rules not processed for user `9bcff29f0f0d444ebae077deb2ff04c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:41.606554 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-177f0933-4866-4b03-a66d-27646f196b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:41.670897 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1311/2621] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:53:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:41.675383 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537190a6-bf9c-48cd-9713-77d5e552706d admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:41.736897 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537190a6-bf9c-48cd-9713-77d5e552706d admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:41.741624 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee900e1-5bfb-401c-b1b7-cf1529956352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:41.742837 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee900e1-5bfb-401c-b1b7-cf1529956352 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:41.743161 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee900e1-5bfb-401c-b1b7-cf1529956352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:41.743446 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee900e1-5bfb-401c-b1b7-cf1529956352 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:41.744134 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee900e1-5bfb-401c-b1b7-cf1529956352 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:41.746040 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee900e1-5bfb-401c-b1b7-cf1529956352 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:41.771185 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1311/2622] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:53:41 2022] POST /identity/v3/projects => generated 393 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:53:41.775392 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f0933-4866-4b03-a66d-27646f196b6c None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:41.785876 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f0933-4866-4b03-a66d-27646f196b6c None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:41.842237 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb7f97-e0ba-4a57-a3ab-912c45f0c0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:41.842823 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb7f97-e0ba-4a57-a3ab-912c45f0c0c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:41.843242 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb7f97-e0ba-4a57-a3ab-912c45f0c0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:41.843526 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb7f97-e0ba-4a57-a3ab-912c45f0c0c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:41.844211 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb7f97-e0ba-4a57-a3ab-912c45f0c0c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:41.846137 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb7f97-e0ba-4a57-a3ab-912c45f0c0c0 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:41.866160 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1312/2623] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:53:41 2022] POST /identity/v3/users => generated 303 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:53:41.871217 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee900e1-5bfb-401c-b1b7-cf1529956352 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:41.881485 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee900e1-5bfb-401c-b1b7-cf1529956352 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:41.886769 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900bdefd-0c46-42d4-a039-439ec4ced4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:41.887494 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900bdefd-0c46-42d4-a039-439ec4ced4d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:41.887783 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900bdefd-0c46-42d4-a039-439ec4ced4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:41.888089 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900bdefd-0c46-42d4-a039-439ec4ced4d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:41.888819 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900bdefd-0c46-42d4-a039-439ec4ced4d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:41.890438 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900bdefd-0c46-42d4-a039-439ec4ced4d9 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:41.900965 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1312/2624] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:53:41.905262 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb7f97-e0ba-4a57-a3ab-912c45f0c0c0 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:41.915745 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb7f97-e0ba-4a57-a3ab-912c45f0c0c0 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:41.920473 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b15ae-9499-4c47-8dc6-392cad98b033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:41.921372 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b15ae-9499-4c47-8dc6-392cad98b033 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:41.921656 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b15ae-9499-4c47-8dc6-392cad98b033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:41.921939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b15ae-9499-4c47-8dc6-392cad98b033 admin admin] PATH_INFO: `/v3/projects/9f771b7402344234bc6e6653b3360e1f/users/63e07cb8f89a47e7b03b6593bb5d8be9/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:41.936893 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7b15ae-9499-4c47-8dc6-392cad98b033 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f771b7402344234bc6e6653b3360e1f, user_id=63e07cb8f89a47e7b03b6593bb5d8be9, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:41.939579 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7b15ae-9499-4c47-8dc6-392cad98b033 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:41.952632 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1313/2625] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:53:41 2022] PUT /identity/v3/projects/9f771b7402344234bc6e6653b3360e1f/users/63e07cb8f89a47e7b03b6593bb5d8be9/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:41.956653 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900bdefd-0c46-42d4-a039-439ec4ced4d9 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:41.966512 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900bdefd-0c46-42d4-a039-439ec4ced4d9 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:41.986399 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:42.026974 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58a0010-acdd-4a1e-a81e-4811ebe6330b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:42.027673 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a0010-acdd-4a1e-a81e-4811ebe6330b admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:42.027957 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a0010-acdd-4a1e-a81e-4811ebe6330b admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:42.028294 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a0010-acdd-4a1e-a81e-4811ebe6330b admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:42.028957 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58a0010-acdd-4a1e-a81e-4811ebe6330b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:42.030527 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58a0010-acdd-4a1e-a81e-4811ebe6330b admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:42.040610 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1313/2626] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:41 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:53:42.044730 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b15ae-9499-4c47-8dc6-392cad98b033 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:42.050189 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:42.057254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b15ae-9499-4c47-8dc6-392cad98b033 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:42.062284 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b279ea0-fad8-471f-91e9-878b44221119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:42.063225 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b279ea0-fad8-471f-91e9-878b44221119 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:42.063505 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b279ea0-fad8-471f-91e9-878b44221119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:42.063781 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b279ea0-fad8-471f-91e9-878b44221119 admin admin] PATH_INFO: `/v3/projects/9f771b7402344234bc6e6653b3360e1f/users/63e07cb8f89a47e7b03b6593bb5d8be9/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:42.065718 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b279ea0-fad8-471f-91e9-878b44221119 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f771b7402344234bc6e6653b3360e1f, user_id=63e07cb8f89a47e7b03b6593bb5d8be9, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:42.068402 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b279ea0-fad8-471f-91e9-878b44221119 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:42.079047 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1314/2627] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:53:42 2022] PUT /identity/v3/projects/9f771b7402344234bc6e6653b3360e1f/users/63e07cb8f89a47e7b03b6593bb5d8be9/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:42.089057 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814755c-850c-44c9-b791-3ea66aabc811 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:42.089526 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814755c-850c-44c9-b791-3ea66aabc811 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:42.089627 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814755c-850c-44c9-b791-3ea66aabc811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:42.116329 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-f814755c-850c-44c9-b791-3ea66aabc811 None None] MFA Rules not processed for user `63e07cb8f89a47e7b03b6593bb5d8be9`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:42.117509 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f814755c-850c-44c9-b791-3ea66aabc811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:42.195735 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1314/2628] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:53:42 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:42.206317 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b279ea0-fad8-471f-91e9-878b44221119 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:42.208223 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6b279ea0-fad8-471f-91e9-878b44221119 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:42.263928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b279ea0-fad8-471f-91e9-878b44221119 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:42.267931 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7aefa2-d51c-43cb-99b7-1ff543aa6277 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:42.268578 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7aefa2-d51c-43cb-99b7-1ff543aa6277 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:42.268861 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7aefa2-d51c-43cb-99b7-1ff543aa6277 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:42.269146 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7aefa2-d51c-43cb-99b7-1ff543aa6277 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:42.278186 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7aefa2-d51c-43cb-99b7-1ff543aa6277 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:42.279658 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7aefa2-d51c-43cb-99b7-1ff543aa6277 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:42.289171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1315/2629] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:42 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:42.337339 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814755c-850c-44c9-b791-3ea66aabc811 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:42.338728 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f814755c-850c-44c9-b791-3ea66aabc811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:42.350575 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814755c-850c-44c9-b791-3ea66aabc811 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:42.354708 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec9510-83fe-4466-88ee-fc1d6725b98e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:42.355388 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec9510-83fe-4466-88ee-fc1d6725b98e service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:42.355679 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec9510-83fe-4466-88ee-fc1d6725b98e service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:42.355960 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec9510-83fe-4466-88ee-fc1d6725b98e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:42.412635 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ec9510-83fe-4466-88ee-fc1d6725b98e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:42.414264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ec9510-83fe-4466-88ee-fc1d6725b98e service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:42.423948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1315/2630] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:50.904698 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7aefa2-d51c-43cb-99b7-1ff543aa6277 service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:50.965873 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7aefa2-d51c-43cb-99b7-1ff543aa6277 service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:50.970436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a31cbe-dd26-4bbf-a2eb-02364bb00f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:50.971326 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a31cbe-dd26-4bbf-a2eb-02364bb00f37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:50.971608 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a31cbe-dd26-4bbf-a2eb-02364bb00f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:50.971881 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a31cbe-dd26-4bbf-a2eb-02364bb00f37 admin admin] PATH_INFO: `/v3/users/9bcff29f0f0d444ebae077deb2ff04c4` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:50.973047 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a31cbe-dd26-4bbf-a2eb-02364bb00f37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bcff29f0f0d444ebae077deb2ff04c4)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:50.974986 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a31cbe-dd26-4bbf-a2eb-02364bb00f37 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:51.021283 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f8a31cbe-dd26-4bbf-a2eb-02364bb00f37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bcff29f0f0d444ebae077deb2ff04c4'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:51.021949 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f8a31cbe-dd26-4bbf-a2eb-02364bb00f37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bcff29f0f0d444ebae077deb2ff04c4'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:51.034044 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f8a31cbe-dd26-4bbf-a2eb-02364bb00f37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bcff29f0f0d444ebae077deb2ff04c4'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:51.034667 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f8a31cbe-dd26-4bbf-a2eb-02364bb00f37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bcff29f0f0d444ebae077deb2ff04c4'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:51.041909 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f8a31cbe-dd26-4bbf-a2eb-02364bb00f37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bcff29f0f0d444ebae077deb2ff04c4'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:51.053810 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1316/2631] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:53:50 2022] DELETE /identity/v3/users/9bcff29f0f0d444ebae077deb2ff04c4 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:51.058494 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29aed0-1eb3-4b7e-b39a-44b975d14145 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:51.058946 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29aed0-1eb3-4b7e-b39a-44b975d14145 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:51.059113 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29aed0-1eb3-4b7e-b39a-44b975d14145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:51.076370 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-cd29aed0-1eb3-4b7e-b39a-44b975d14145 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:51.077563 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-cd29aed0-1eb3-4b7e-b39a-44b975d14145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:51.116964 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:51.136058 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1316/2632] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:53:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:51.146049 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a31cbe-dd26-4bbf-a2eb-02364bb00f37 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:51.147489 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f8a31cbe-dd26-4bbf-a2eb-02364bb00f37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:51.161839 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:51.207673 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a31cbe-dd26-4bbf-a2eb-02364bb00f37 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:51.211683 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb0b422-be03-42e0-a0a1-e136cad3657a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:51.212903 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0b422-be03-42e0-a0a1-e136cad3657a service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:51.213188 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0b422-be03-42e0-a0a1-e136cad3657a service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:51.213465 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0b422-be03-42e0-a0a1-e136cad3657a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:51.224030 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb0b422-be03-42e0-a0a1-e136cad3657a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:51.225459 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb0b422-be03-42e0-a0a1-e136cad3657a service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:51.235040 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1317/2633] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:51.507217 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29aed0-1eb3-4b7e-b39a-44b975d14145 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:51.508732 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-cd29aed0-1eb3-4b7e-b39a-44b975d14145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:51.522525 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29aed0-1eb3-4b7e-b39a-44b975d14145 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:51.527525 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445d6fb-f794-4006-b966-d3a9cf9688a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:51.528466 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445d6fb-f794-4006-b966-d3a9cf9688a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:51.528787 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445d6fb-f794-4006-b966-d3a9cf9688a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:51.529066 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445d6fb-f794-4006-b966-d3a9cf9688a8 admin admin] PATH_INFO: `/v3/projects/339dfd7ff18d454ea37a9c4b4f41885c` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:51.530258 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d445d6fb-f794-4006-b966-d3a9cf9688a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=339dfd7ff18d454ea37a9c4b4f41885c)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:51.532491 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d445d6fb-f794-4006-b966-d3a9cf9688a8 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:51.593075 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-d445d6fb-f794-4006-b966-d3a9cf9688a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '339dfd7ff18d454ea37a9c4b4f41885c', 'request_id': 'req-d445d6fb-f794-4006-b966-d3a9cf9688a8'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:51.601421 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-d445d6fb-f794-4006-b966-d3a9cf9688a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '339dfd7ff18d454ea37a9c4b4f41885c', 'request_id': 'req-d445d6fb-f794-4006-b966-d3a9cf9688a8'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:51.610196 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-d445d6fb-f794-4006-b966-d3a9cf9688a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '339dfd7ff18d454ea37a9c4b4f41885c', 'request_id': 'req-d445d6fb-f794-4006-b966-d3a9cf9688a8'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:51.622374 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-d445d6fb-f794-4006-b966-d3a9cf9688a8 admin admin] The token cache is being invalidate because project 339dfd7ff18d454ea37a9c4b4f41885c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:53:51.622791 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-d445d6fb-f794-4006-b966-d3a9cf9688a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:51.623502 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-d445d6fb-f794-4006-b966-d3a9cf9688a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:51.625055 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1317/2634] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:53:51 2022] DELETE /identity/v3/projects/339dfd7ff18d454ea37a9c4b4f41885c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:51.628950 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb0b422-be03-42e0-a0a1-e136cad3657a service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:51.637238 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445d6fb-f794-4006-b966-d3a9cf9688a8 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:51.638766 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d445d6fb-f794-4006-b966-d3a9cf9688a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:51.639880 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb0b422-be03-42e0-a0a1-e136cad3657a service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:51.701758 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e16a4-7947-4ce6-95ab-fee56bf3e8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:51.702613 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445d6fb-f794-4006-b966-d3a9cf9688a8 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:51.702780 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e16a4-7947-4ce6-95ab-fee56bf3e8a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:51.702918 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e16a4-7947-4ce6-95ab-fee56bf3e8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:51.703208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e16a4-7947-4ce6-95ab-fee56bf3e8a6 admin admin] PATH_INFO: `/v3/users/63e07cb8f89a47e7b03b6593bb5d8be9` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:51.707131 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e16a4-7947-4ce6-95ab-fee56bf3e8a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63e07cb8f89a47e7b03b6593bb5d8be9)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:51.709048 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e16a4-7947-4ce6-95ab-fee56bf3e8a6 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:51.709422 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be04d60a-b584-486d-b64d-db3bb448ee24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:51.710071 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04d60a-b584-486d-b64d-db3bb448ee24 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:51.710356 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04d60a-b584-486d-b64d-db3bb448ee24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:51.710635 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04d60a-b584-486d-b64d-db3bb448ee24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:51.712489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-be04d60a-b584-486d-b64d-db3bb448ee24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:51.722636 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be04d60a-b584-486d-b64d-db3bb448ee24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:51.724118 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be04d60a-b584-486d-b64d-db3bb448ee24 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:51.732758 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1318/2635] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:51.768707 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-af9e16a4-7947-4ce6-95ab-fee56bf3e8a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63e07cb8f89a47e7b03b6593bb5d8be9'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:51.769390 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-af9e16a4-7947-4ce6-95ab-fee56bf3e8a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63e07cb8f89a47e7b03b6593bb5d8be9'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:51.782478 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-af9e16a4-7947-4ce6-95ab-fee56bf3e8a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63e07cb8f89a47e7b03b6593bb5d8be9'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:51.783216 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-af9e16a4-7947-4ce6-95ab-fee56bf3e8a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63e07cb8f89a47e7b03b6593bb5d8be9'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:51.791076 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-af9e16a4-7947-4ce6-95ab-fee56bf3e8a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63e07cb8f89a47e7b03b6593bb5d8be9'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:51.803595 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1318/2636] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:53:51 2022] DELETE /identity/v3/users/63e07cb8f89a47e7b03b6593bb5d8be9 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:52.172389 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be04d60a-b584-486d-b64d-db3bb448ee24 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:52.173926 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-be04d60a-b584-486d-b64d-db3bb448ee24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:52.193229 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be04d60a-b584-486d-b64d-db3bb448ee24 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:52.257207 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31beebf8-c149-459c-8567-36f8532e0f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:52.258072 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31beebf8-c149-459c-8567-36f8532e0f76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:52.258359 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31beebf8-c149-459c-8567-36f8532e0f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:52.258640 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31beebf8-c149-459c-8567-36f8532e0f76 admin admin] PATH_INFO: `/v3/projects/9f771b7402344234bc6e6653b3360e1f` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:52.260608 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31beebf8-c149-459c-8567-36f8532e0f76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f771b7402344234bc6e6653b3360e1f)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:52.262390 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31beebf8-c149-459c-8567-36f8532e0f76 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:52.273990 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:52.311471 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-31beebf8-c149-459c-8567-36f8532e0f76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f771b7402344234bc6e6653b3360e1f', 'request_id': 'req-31beebf8-c149-459c-8567-36f8532e0f76'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:52.320212 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-31beebf8-c149-459c-8567-36f8532e0f76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f771b7402344234bc6e6653b3360e1f', 'request_id': 'req-31beebf8-c149-459c-8567-36f8532e0f76'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:52.327910 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-31beebf8-c149-459c-8567-36f8532e0f76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f771b7402344234bc6e6653b3360e1f', 'request_id': 'req-31beebf8-c149-459c-8567-36f8532e0f76'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:52.338754 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-31beebf8-c149-459c-8567-36f8532e0f76 admin admin] The token cache is being invalidate because project 9f771b7402344234bc6e6653b3360e1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:53:52.339441 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-31beebf8-c149-459c-8567-36f8532e0f76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:52.339738 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-31beebf8-c149-459c-8567-36f8532e0f76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:52.341300 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1319/2637] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:53:52 2022] DELETE /identity/v3/projects/9f771b7402344234bc6e6653b3360e1f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:52.346351 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981f5de-0d88-4d31-8436-f69695a50e8b None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:52.346820 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981f5de-0d88-4d31-8436-f69695a50e8b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:52.346919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981f5de-0d88-4d31-8436-f69695a50e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:52.364426 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-a981f5de-0d88-4d31-8436-f69695a50e8b None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:52.365656 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a981f5de-0d88-4d31-8436-f69695a50e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:52.396956 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:52.432644 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1319/2638] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:53:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:52.440771 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee4888-3f36-4b5c-b4d1-55fb12e5a6c2 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:52.440948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee4888-3f36-4b5c-b4d1-55fb12e5a6c2 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:52.441126 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee4888-3f36-4b5c-b4d1-55fb12e5a6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:52.460243 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-47ee4888-3f36-4b5c-b4d1-55fb12e5a6c2 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:52.461500 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-47ee4888-3f36-4b5c-b4d1-55fb12e5a6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:52.469603 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1320/2639] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:53:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:52.473987 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a981f5de-0d88-4d31-8436-f69695a50e8b None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:52.484318 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a981f5de-0d88-4d31-8436-f69695a50e8b None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:52.490292 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad542d6-b155-488e-94e0-49cd52fb2a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:52.491034 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad542d6-b155-488e-94e0-49cd52fb2a1f admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:52.491339 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad542d6-b155-488e-94e0-49cd52fb2a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:52.491639 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad542d6-b155-488e-94e0-49cd52fb2a1f admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:52.492437 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad542d6-b155-488e-94e0-49cd52fb2a1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:53:52.492745 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad542d6-b155-488e-94e0-49cd52fb2a1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:52.494960 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad542d6-b155-488e-94e0-49cd52fb2a1f admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:52.506638 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1320/2640] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:53:52.511529 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee4888-3f36-4b5c-b4d1-55fb12e5a6c2 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:52.521081 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee4888-3f36-4b5c-b4d1-55fb12e5a6c2 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:52.525891 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d643857-f37a-4c0a-947e-c6a467e46e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:52.526551 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d643857-f37a-4c0a-947e-c6a467e46e7f admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:52.526840 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d643857-f37a-4c0a-947e-c6a467e46e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:52.527159 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d643857-f37a-4c0a-947e-c6a467e46e7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:52.527854 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d643857-f37a-4c0a-947e-c6a467e46e7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:52.529759 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d643857-f37a-4c0a-947e-c6a467e46e7f admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:52.555311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1321/2641] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:53:52 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:53:52.559914 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad542d6-b155-488e-94e0-49cd52fb2a1f admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:52.569863 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad542d6-b155-488e-94e0-49cd52fb2a1f admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:52.626619 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f55a7d-30a5-4522-a151-03ba246c7c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:52.627340 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f55a7d-30a5-4522-a151-03ba246c7c95 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:52.627628 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f55a7d-30a5-4522-a151-03ba246c7c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:52.627910 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f55a7d-30a5-4522-a151-03ba246c7c95 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:52.628622 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f55a7d-30a5-4522-a151-03ba246c7c95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:52.630621 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f55a7d-30a5-4522-a151-03ba246c7c95 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:52.650495 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1321/2642] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:53:52 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:53:52.654562 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d643857-f37a-4c0a-947e-c6a467e46e7f admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:52.665795 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d643857-f37a-4c0a-947e-c6a467e46e7f admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:52.670493 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e23413-1e28-4ccd-9911-9b1b53a71fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:52.671196 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e23413-1e28-4ccd-9911-9b1b53a71fde admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:52.671487 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e23413-1e28-4ccd-9911-9b1b53a71fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:52.671784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e23413-1e28-4ccd-9911-9b1b53a71fde admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:52.672499 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e23413-1e28-4ccd-9911-9b1b53a71fde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:52.674092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e23413-1e28-4ccd-9911-9b1b53a71fde admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:52.687038 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1322/2643] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:53:52.691907 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f55a7d-30a5-4522-a151-03ba246c7c95 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:52.700991 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f55a7d-30a5-4522-a151-03ba246c7c95 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:52.706399 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33455edc-522a-47ab-8a2a-8e2ec39e70a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:52.707232 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33455edc-522a-47ab-8a2a-8e2ec39e70a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:52.707625 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33455edc-522a-47ab-8a2a-8e2ec39e70a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:52.708082 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33455edc-522a-47ab-8a2a-8e2ec39e70a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:52.708960 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33455edc-522a-47ab-8a2a-8e2ec39e70a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:52.710518 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33455edc-522a-47ab-8a2a-8e2ec39e70a1 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:52.721374 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1322/2644] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:53:52.728516 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e23413-1e28-4ccd-9911-9b1b53a71fde admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:52.738521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e23413-1e28-4ccd-9911-9b1b53a71fde admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:52.743172 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e02768-26bd-4a2c-a401-81d1de8abc76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:52.744638 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e02768-26bd-4a2c-a401-81d1de8abc76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:52.744926 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e02768-26bd-4a2c-a401-81d1de8abc76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:52.745233 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e02768-26bd-4a2c-a401-81d1de8abc76 admin admin] PATH_INFO: `/v3/projects/ef7726e2c89148cfb86970c5da946e22/users/aaa7883a746646d28336f275b1891718/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:52.759708 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e02768-26bd-4a2c-a401-81d1de8abc76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef7726e2c89148cfb86970c5da946e22, user_id=aaa7883a746646d28336f275b1891718, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:52.762442 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e02768-26bd-4a2c-a401-81d1de8abc76 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:52.775382 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1323/2645] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:53:52 2022] PUT /identity/v3/projects/ef7726e2c89148cfb86970c5da946e22/users/aaa7883a746646d28336f275b1891718/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:52.785573 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6dea23-ea6b-4aef-ac6c-73722412ed91 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:52.785869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6dea23-ea6b-4aef-ac6c-73722412ed91 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:52.786149 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6dea23-ea6b-4aef-ac6c-73722412ed91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:52.813379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-bb6dea23-ea6b-4aef-ac6c-73722412ed91 None None] MFA Rules not processed for user `aaa7883a746646d28336f275b1891718`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:52.814814 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-bb6dea23-ea6b-4aef-ac6c-73722412ed91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:52.877590 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1323/2646] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:53:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:52.888653 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e02768-26bd-4a2c-a401-81d1de8abc76 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:52.890356 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f7e02768-26bd-4a2c-a401-81d1de8abc76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:52.948087 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e02768-26bd-4a2c-a401-81d1de8abc76 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:52.953188 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e38ff-1b82-48b7-9fb6-d33ab1232265 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:52.953380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e38ff-1b82-48b7-9fb6-d33ab1232265 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:52.953672 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e38ff-1b82-48b7-9fb6-d33ab1232265 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:52.953974 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e38ff-1b82-48b7-9fb6-d33ab1232265 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:52.963868 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e38ff-1b82-48b7-9fb6-d33ab1232265 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:52.965310 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e38ff-1b82-48b7-9fb6-d33ab1232265 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:52.974411 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1324/2647] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:52.995015 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6dea23-ea6b-4aef-ac6c-73722412ed91 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:53.025586 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:53.059513 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6dea23-ea6b-4aef-ac6c-73722412ed91 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:53.064138 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e607dd-acd8-48b4-9ec1-aa1bf4cbabf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:53.065015 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e607dd-acd8-48b4-9ec1-aa1bf4cbabf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:53.065306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e607dd-acd8-48b4-9ec1-aa1bf4cbabf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:53.065590 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e607dd-acd8-48b4-9ec1-aa1bf4cbabf4 admin admin] PATH_INFO: `/v3/users/aaa7883a746646d28336f275b1891718` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:53.066914 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e607dd-acd8-48b4-9ec1-aa1bf4cbabf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaa7883a746646d28336f275b1891718)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:53.068996 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e607dd-acd8-48b4-9ec1-aa1bf4cbabf4 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:53.115765 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-c5e607dd-acd8-48b4-9ec1-aa1bf4cbabf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaa7883a746646d28336f275b1891718'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:53.116424 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-c5e607dd-acd8-48b4-9ec1-aa1bf4cbabf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaa7883a746646d28336f275b1891718'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:53.128345 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-c5e607dd-acd8-48b4-9ec1-aa1bf4cbabf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaa7883a746646d28336f275b1891718'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:53.128931 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-c5e607dd-acd8-48b4-9ec1-aa1bf4cbabf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaa7883a746646d28336f275b1891718'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:53.136123 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-c5e607dd-acd8-48b4-9ec1-aa1bf4cbabf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaa7883a746646d28336f275b1891718'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:53.148261 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1324/2648] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:53:52 2022] DELETE /identity/v3/users/aaa7883a746646d28336f275b1891718 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:53.153379 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0b89b-39f1-498d-b02c-1021963eb3a3 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:53.153670 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0b89b-39f1-498d-b02c-1021963eb3a3 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:53.153977 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0b89b-39f1-498d-b02c-1021963eb3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:53.171843 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-37c0b89b-39f1-498d-b02c-1021963eb3a3 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:53.173046 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-37c0b89b-39f1-498d-b02c-1021963eb3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:53.205713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:53.234697 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1325/2649] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:53:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:53.244654 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e607dd-acd8-48b4-9ec1-aa1bf4cbabf4 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:53.246124 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c5e607dd-acd8-48b4-9ec1-aa1bf4cbabf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:53.305549 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e607dd-acd8-48b4-9ec1-aa1bf4cbabf4 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:53.309787 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d43d2ef-0829-4e17-84ed-dccf826d456c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:53.310437 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43d2ef-0829-4e17-84ed-dccf826d456c service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:53.310726 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43d2ef-0829-4e17-84ed-dccf826d456c service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:53.311033 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43d2ef-0829-4e17-84ed-dccf826d456c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:53.320092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d43d2ef-0829-4e17-84ed-dccf826d456c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:53.322098 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d43d2ef-0829-4e17-84ed-dccf826d456c service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:53.332093 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1325/2650] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:53:53.606480 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0b89b-39f1-498d-b02c-1021963eb3a3 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:53.608033 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-37c0b89b-39f1-498d-b02c-1021963eb3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:53.623125 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0b89b-39f1-498d-b02c-1021963eb3a3 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:53.627767 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1dec47-743c-4ed4-b8fb-9d4b21640d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:53.628628 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1dec47-743c-4ed4-b8fb-9d4b21640d4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:53.628911 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1dec47-743c-4ed4-b8fb-9d4b21640d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:53.629190 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1dec47-743c-4ed4-b8fb-9d4b21640d4a admin admin] PATH_INFO: `/v3/projects/ef7726e2c89148cfb86970c5da946e22` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:53.630407 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1dec47-743c-4ed4-b8fb-9d4b21640d4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef7726e2c89148cfb86970c5da946e22)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:53.632355 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1dec47-743c-4ed4-b8fb-9d4b21640d4a admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:53.684943 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-1a1dec47-743c-4ed4-b8fb-9d4b21640d4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef7726e2c89148cfb86970c5da946e22', 'request_id': 'req-1a1dec47-743c-4ed4-b8fb-9d4b21640d4a'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:53.692263 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-1a1dec47-743c-4ed4-b8fb-9d4b21640d4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef7726e2c89148cfb86970c5da946e22', 'request_id': 'req-1a1dec47-743c-4ed4-b8fb-9d4b21640d4a'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:53.698820 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-1a1dec47-743c-4ed4-b8fb-9d4b21640d4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef7726e2c89148cfb86970c5da946e22', 'request_id': 'req-1a1dec47-743c-4ed4-b8fb-9d4b21640d4a'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:53.709361 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-1a1dec47-743c-4ed4-b8fb-9d4b21640d4a admin admin] The token cache is being invalidate because project ef7726e2c89148cfb86970c5da946e22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:53:53.709646 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-1a1dec47-743c-4ed4-b8fb-9d4b21640d4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:53.710288 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-1a1dec47-743c-4ed4-b8fb-9d4b21640d4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:53.711866 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1326/2651] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:53:53 2022] DELETE /identity/v3/projects/ef7726e2c89148cfb86970c5da946e22 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:53.717904 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2616c-dd76-41c7-9950-e07a0bb9ca46 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:53.718484 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2616c-dd76-41c7-9950-e07a0bb9ca46 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:53.718484 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2616c-dd76-41c7-9950-e07a0bb9ca46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:53.738803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-6ca2616c-dd76-41c7-9950-e07a0bb9ca46 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:53.740165 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6ca2616c-dd76-41c7-9950-e07a0bb9ca46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:53.786933 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:53.798700 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1dec47-743c-4ed4-b8fb-9d4b21640d4a admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:53.800300 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1a1dec47-743c-4ed4-b8fb-9d4b21640d4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:53.805285 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1326/2652] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:53:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:53.814649 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1682e5b6-cb02-4578-a9a4-19e352ee7787 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:53.814941 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1682e5b6-cb02-4578-a9a4-19e352ee7787 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:53.815242 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1682e5b6-cb02-4578-a9a4-19e352ee7787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:53.834793 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-1682e5b6-cb02-4578-a9a4-19e352ee7787 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:53.836030 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1682e5b6-cb02-4578-a9a4-19e352ee7787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:53.844468 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1327/2653] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:53:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:53.848575 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1682e5b6-cb02-4578-a9a4-19e352ee7787 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:53.858238 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1682e5b6-cb02-4578-a9a4-19e352ee7787 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:53.858987 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1dec47-743c-4ed4-b8fb-9d4b21640d4a admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:53.862641 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64224be-ce95-465c-a210-e6c9231ee190 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:53.862961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d327ec-cb17-4268-8e55-69113c45effe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:53.863318 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64224be-ce95-465c-a210-e6c9231ee190 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:53.863605 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64224be-ce95-465c-a210-e6c9231ee190 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:53.863716 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d327ec-cb17-4268-8e55-69113c45effe admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:53.863885 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64224be-ce95-465c-a210-e6c9231ee190 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:53.864008 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d327ec-cb17-4268-8e55-69113c45effe admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:53.864191 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d327ec-cb17-4268-8e55-69113c45effe admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:53.864903 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d327ec-cb17-4268-8e55-69113c45effe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:53:53.865208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d327ec-cb17-4268-8e55-69113c45effe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:53.865761 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c64224be-ce95-465c-a210-e6c9231ee190 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:53.866843 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d327ec-cb17-4268-8e55-69113c45effe admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:53.878148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1328/2654] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:53:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:53:53.882394 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d327ec-cb17-4268-8e55-69113c45effe admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:53.891942 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d327ec-cb17-4268-8e55-69113c45effe admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:53.896622 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccbb175-1f07-4b65-a586-8a605b2de505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:53.897252 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccbb175-1f07-4b65-a586-8a605b2de505 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:53.897533 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccbb175-1f07-4b65-a586-8a605b2de505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:53.897810 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccbb175-1f07-4b65-a586-8a605b2de505 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:53.898481 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccbb175-1f07-4b65-a586-8a605b2de505 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:53.900963 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccbb175-1f07-4b65-a586-8a605b2de505 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:53.928373 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1329/2655] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:53:53 2022] POST /identity/v3/projects => generated 359 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:53:53.932334 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccbb175-1f07-4b65-a586-8a605b2de505 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:53.940252 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64224be-ce95-465c-a210-e6c9231ee190 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:53.941703 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64224be-ce95-465c-a210-e6c9231ee190 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:53.952933 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1327/2656] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:53:53.993594 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccbb175-1f07-4b65-a586-8a605b2de505 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:53.998401 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aee715-59a5-494a-9c8b-bab5424f3eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:53.999134 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aee715-59a5-494a-9c8b-bab5424f3eae admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:53.999423 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aee715-59a5-494a-9c8b-bab5424f3eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:53.999702 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aee715-59a5-494a-9c8b-bab5424f3eae admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:54.000392 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aee715-59a5-494a-9c8b-bab5424f3eae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:54.002350 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aee715-59a5-494a-9c8b-bab5424f3eae admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:54.023205 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1330/2657] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:53:53 2022] POST /identity/v3/users => generated 280 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:53:54.027773 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64224be-ce95-465c-a210-e6c9231ee190 service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:54.034182 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:54.039116 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64224be-ce95-465c-a210-e6c9231ee190 service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:54.045977 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab05c81c-5786-4d2c-8037-68549ba09c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:54.047122 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05c81c-5786-4d2c-8037-68549ba09c20 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:54.047593 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05c81c-5786-4d2c-8037-68549ba09c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:54.047929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05c81c-5786-4d2c-8037-68549ba09c20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:54.048877 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab05c81c-5786-4d2c-8037-68549ba09c20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:54.050503 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab05c81c-5786-4d2c-8037-68549ba09c20 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:54.064548 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1328/2658] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:53:54.065899 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aee715-59a5-494a-9c8b-bab5424f3eae admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:54.077444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aee715-59a5-494a-9c8b-bab5424f3eae admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:54.082390 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c533506-926c-40ea-825b-39d7b6f9a6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:54.083087 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c533506-926c-40ea-825b-39d7b6f9a6b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:54.083369 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c533506-926c-40ea-825b-39d7b6f9a6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:54.083711 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c533506-926c-40ea-825b-39d7b6f9a6b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:54.084440 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c533506-926c-40ea-825b-39d7b6f9a6b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:54.086106 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c533506-926c-40ea-825b-39d7b6f9a6b2 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:54.098906 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1331/2659] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:53:54.103917 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab05c81c-5786-4d2c-8037-68549ba09c20 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:54.114346 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab05c81c-5786-4d2c-8037-68549ba09c20 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:54.119481 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d32ac-1926-4400-9eed-6cfd4eb404c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:54.120381 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d32ac-1926-4400-9eed-6cfd4eb404c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:54.120669 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d32ac-1926-4400-9eed-6cfd4eb404c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:54.120951 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d32ac-1926-4400-9eed-6cfd4eb404c8 admin admin] PATH_INFO: `/v3/projects/591cd386f539470cb36e8a625317f0e8/users/e109d45dfc45403d8555ae0ddfc1656f/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:54.140983 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952d32ac-1926-4400-9eed-6cfd4eb404c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=591cd386f539470cb36e8a625317f0e8, user_id=e109d45dfc45403d8555ae0ddfc1656f, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:54.143740 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952d32ac-1926-4400-9eed-6cfd4eb404c8 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:54.158367 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1329/2660] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:53:54 2022] PUT /identity/v3/projects/591cd386f539470cb36e8a625317f0e8/users/e109d45dfc45403d8555ae0ddfc1656f/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:54.169402 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829e8b3-7ad0-4930-bbce-0552ef570846 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:54.169402 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829e8b3-7ad0-4930-bbce-0552ef570846 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:54.169795 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829e8b3-7ad0-4930-bbce-0552ef570846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:54.197091 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-5829e8b3-7ad0-4930-bbce-0552ef570846 None None] MFA Rules not processed for user `e109d45dfc45403d8555ae0ddfc1656f`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:54.198583 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5829e8b3-7ad0-4930-bbce-0552ef570846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:54.272810 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1332/2661] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:53:54 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:54.303569 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d32ac-1926-4400-9eed-6cfd4eb404c8 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:54.305360 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-952d32ac-1926-4400-9eed-6cfd4eb404c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:54.362101 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d32ac-1926-4400-9eed-6cfd4eb404c8 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:54.365808 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4a1a1-f957-4acf-8a0f-d229a6e1a8d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:54.366450 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4a1a1-f957-4acf-8a0f-d229a6e1a8d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:54.366739 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4a1a1-f957-4acf-8a0f-d229a6e1a8d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:54.367052 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4a1a1-f957-4acf-8a0f-d229a6e1a8d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:54.375427 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be4a1a1-f957-4acf-8a0f-d229a6e1a8d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:54.376861 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be4a1a1-f957-4acf-8a0f-d229a6e1a8d7 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:54.386439 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1330/2662] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:54 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:54.401623 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5829e8b3-7ad0-4930-bbce-0552ef570846 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:54.420458 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:54.466336 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5829e8b3-7ad0-4930-bbce-0552ef570846 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:54.470826 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2e43e-579a-4d00-9251-affc12f16eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:54.471724 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2e43e-579a-4d00-9251-affc12f16eea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:54.472005 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2e43e-579a-4d00-9251-affc12f16eea admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:54.472287 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2e43e-579a-4d00-9251-affc12f16eea admin admin] PATH_INFO: `/v3/users/e109d45dfc45403d8555ae0ddfc1656f` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:54.473491 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe2e43e-579a-4d00-9251-affc12f16eea admin admin] RBAC: Authorizing `identity:delete_user(user_id=e109d45dfc45403d8555ae0ddfc1656f)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:54.475448 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe2e43e-579a-4d00-9251-affc12f16eea admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:54.521952 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-bfe2e43e-579a-4d00-9251-affc12f16eea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e109d45dfc45403d8555ae0ddfc1656f'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:54.523304 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-bfe2e43e-579a-4d00-9251-affc12f16eea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e109d45dfc45403d8555ae0ddfc1656f'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:54.535140 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-bfe2e43e-579a-4d00-9251-affc12f16eea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e109d45dfc45403d8555ae0ddfc1656f'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:54.535823 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-bfe2e43e-579a-4d00-9251-affc12f16eea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e109d45dfc45403d8555ae0ddfc1656f'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:54.543790 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-bfe2e43e-579a-4d00-9251-affc12f16eea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e109d45dfc45403d8555ae0ddfc1656f'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:54.556544 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1333/2663] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:53:54 2022] DELETE /identity/v3/users/e109d45dfc45403d8555ae0ddfc1656f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:54.561111 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7beb1f-8c75-4177-ae68-95bd696a70a1 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:54.561401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7beb1f-8c75-4177-ae68-95bd696a70a1 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:54.561680 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7beb1f-8c75-4177-ae68-95bd696a70a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:54.579129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-5f7beb1f-8c75-4177-ae68-95bd696a70a1 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:54.580401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-5f7beb1f-8c75-4177-ae68-95bd696a70a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:54.640276 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1331/2664] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:53:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:53:54.649887 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2e43e-579a-4d00-9251-affc12f16eea admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:54.651326 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-bfe2e43e-579a-4d00-9251-affc12f16eea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:54.708866 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2e43e-579a-4d00-9251-affc12f16eea admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:54.712630 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9779d9d5-6dd0-4510-b4c3-d83df51a0adc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:54.713279 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779d9d5-6dd0-4510-b4c3-d83df51a0adc service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:54.713567 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779d9d5-6dd0-4510-b4c3-d83df51a0adc service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:54.713847 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779d9d5-6dd0-4510-b4c3-d83df51a0adc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:54.723171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9779d9d5-6dd0-4510-b4c3-d83df51a0adc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:54.724556 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9779d9d5-6dd0-4510-b4c3-d83df51a0adc service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:54.734352 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1334/2665] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:55.002217 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7beb1f-8c75-4177-ae68-95bd696a70a1 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:55.003747 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-5f7beb1f-8c75-4177-ae68-95bd696a70a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:55.015829 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7beb1f-8c75-4177-ae68-95bd696a70a1 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:55.020845 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0c88a-26c3-4c7c-a395-de7766f487b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:55.021710 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0c88a-26c3-4c7c-a395-de7766f487b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:55.021995 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0c88a-26c3-4c7c-a395-de7766f487b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:55.022283 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0c88a-26c3-4c7c-a395-de7766f487b7 admin admin] PATH_INFO: `/v3/projects/591cd386f539470cb36e8a625317f0e8` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:55.023495 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0c88a-26c3-4c7c-a395-de7766f487b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=591cd386f539470cb36e8a625317f0e8)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:55.025401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0c88a-26c3-4c7c-a395-de7766f487b7 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:55.073154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-79c0c88a-26c3-4c7c-a395-de7766f487b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '591cd386f539470cb36e8a625317f0e8', 'request_id': 'req-79c0c88a-26c3-4c7c-a395-de7766f487b7'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:55.080720 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-79c0c88a-26c3-4c7c-a395-de7766f487b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '591cd386f539470cb36e8a625317f0e8', 'request_id': 'req-79c0c88a-26c3-4c7c-a395-de7766f487b7'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:55.086904 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-79c0c88a-26c3-4c7c-a395-de7766f487b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '591cd386f539470cb36e8a625317f0e8', 'request_id': 'req-79c0c88a-26c3-4c7c-a395-de7766f487b7'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:55.091740 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:55.099594 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-79c0c88a-26c3-4c7c-a395-de7766f487b7 admin admin] The token cache is being invalidate because project 591cd386f539470cb36e8a625317f0e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:53:55.100099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-79c0c88a-26c3-4c7c-a395-de7766f487b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:55.100832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-79c0c88a-26c3-4c7c-a395-de7766f487b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:55.102551 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1332/2666] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:53:55 2022] DELETE /identity/v3/projects/591cd386f539470cb36e8a625317f0e8 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:55.108144 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7f599-6004-44b2-a62a-2f56bb50e2fe None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:55.108437 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7f599-6004-44b2-a62a-2f56bb50e2fe None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:55.108742 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7f599-6004-44b2-a62a-2f56bb50e2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:55.130066 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-80c7f599-6004-44b2-a62a-2f56bb50e2fe None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:55.131305 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-80c7f599-6004-44b2-a62a-2f56bb50e2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:55.168832 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:55.199497 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1335/2667] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:53:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:55.207280 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5934491-fd90-44e3-84d5-a9423c13f8d0 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:55.208261 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5934491-fd90-44e3-84d5-a9423c13f8d0 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:55.208887 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5934491-fd90-44e3-84d5-a9423c13f8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:55.233671 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-d5934491-fd90-44e3-84d5-a9423c13f8d0 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:55.235064 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d5934491-fd90-44e3-84d5-a9423c13f8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:55.243257 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1333/2668] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:53:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:55.248314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c7f599-6004-44b2-a62a-2f56bb50e2fe None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:55.258858 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c7f599-6004-44b2-a62a-2f56bb50e2fe None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:55.265018 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992ab12-e7d3-4130-a445-d0f241a2d68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:55.265886 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992ab12-e7d3-4130-a445-d0f241a2d68c admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:55.266178 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992ab12-e7d3-4130-a445-d0f241a2d68c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:55.266453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992ab12-e7d3-4130-a445-d0f241a2d68c admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:55.267229 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1992ab12-e7d3-4130-a445-d0f241a2d68c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:53:55.267573 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1992ab12-e7d3-4130-a445-d0f241a2d68c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:55.270565 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1992ab12-e7d3-4130-a445-d0f241a2d68c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:55.285798 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1336/2669] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:53:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:53:55.286964 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5934491-fd90-44e3-84d5-a9423c13f8d0 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:55.297952 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5934491-fd90-44e3-84d5-a9423c13f8d0 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:55.303078 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e9e4f-c31f-44c6-8628-abe65b0b36d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:55.303739 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e9e4f-c31f-44c6-8628-abe65b0b36d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:55.304280 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e9e4f-c31f-44c6-8628-abe65b0b36d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:55.304280 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e9e4f-c31f-44c6-8628-abe65b0b36d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:55.304967 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117e9e4f-c31f-44c6-8628-abe65b0b36d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:55.306909 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117e9e4f-c31f-44c6-8628-abe65b0b36d8 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:55.346636 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1334/2670] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:53:55 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:53:55.350661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992ab12-e7d3-4130-a445-d0f241a2d68c admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:55.361820 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992ab12-e7d3-4130-a445-d0f241a2d68c admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:55.419904 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb5c66-62d9-4894-81c8-e0629e64d124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:55.420580 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb5c66-62d9-4894-81c8-e0629e64d124 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:55.420871 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb5c66-62d9-4894-81c8-e0629e64d124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:55.421152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb5c66-62d9-4894-81c8-e0629e64d124 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:55.421846 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb5c66-62d9-4894-81c8-e0629e64d124 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:55.423793 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb5c66-62d9-4894-81c8-e0629e64d124 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:55.447118 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1337/2671] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:53:55 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:53:55.451357 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e9e4f-c31f-44c6-8628-abe65b0b36d8 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:55.461855 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e9e4f-c31f-44c6-8628-abe65b0b36d8 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:55.467663 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33214f40-8275-41a3-834b-5acdeb27e3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:55.468335 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33214f40-8275-41a3-834b-5acdeb27e3e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:55.468648 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33214f40-8275-41a3-834b-5acdeb27e3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:55.468926 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33214f40-8275-41a3-834b-5acdeb27e3e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:55.469686 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33214f40-8275-41a3-834b-5acdeb27e3e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:55.471348 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33214f40-8275-41a3-834b-5acdeb27e3e9 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:55.483217 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1335/2672] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:53:55.487257 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb5c66-62d9-4894-81c8-e0629e64d124 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:55.497188 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb5c66-62d9-4894-81c8-e0629e64d124 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:55.501862 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e491d42-e35d-48f4-8af8-47d7786ba860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:55.502591 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e491d42-e35d-48f4-8af8-47d7786ba860 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:55.502879 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e491d42-e35d-48f4-8af8-47d7786ba860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:55.503236 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e491d42-e35d-48f4-8af8-47d7786ba860 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:55.503974 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e491d42-e35d-48f4-8af8-47d7786ba860 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:55.505563 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e491d42-e35d-48f4-8af8-47d7786ba860 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:55.515737 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1338/2673] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:53:55.519798 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33214f40-8275-41a3-834b-5acdeb27e3e9 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:55.530860 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33214f40-8275-41a3-834b-5acdeb27e3e9 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:55.536211 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f8ec2-7096-48cc-8b49-919252ee0f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:55.537091 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f8ec2-7096-48cc-8b49-919252ee0f0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:55.537375 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f8ec2-7096-48cc-8b49-919252ee0f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:55.537653 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f8ec2-7096-48cc-8b49-919252ee0f0a admin admin] PATH_INFO: `/v3/projects/f00730bc0c964ca082ce22d66b613847/users/ec669b302f9b4067b587bea4b75fd918/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:55.552837 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f8ec2-7096-48cc-8b49-919252ee0f0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f00730bc0c964ca082ce22d66b613847, user_id=ec669b302f9b4067b587bea4b75fd918, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:55.555512 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f8ec2-7096-48cc-8b49-919252ee0f0a admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:55.568616 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1336/2674] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:53:55 2022] PUT /identity/v3/projects/f00730bc0c964ca082ce22d66b613847/users/ec669b302f9b4067b587bea4b75fd918/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:53:55.580329 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c806321d-282f-444e-8fba-a4007fea2c3b None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:55.580627 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c806321d-282f-444e-8fba-a4007fea2c3b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:55.580914 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c806321d-282f-444e-8fba-a4007fea2c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:55.607659 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-c806321d-282f-444e-8fba-a4007fea2c3b None None] MFA Rules not processed for user `ec669b302f9b4067b587bea4b75fd918`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:55.608878 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c806321d-282f-444e-8fba-a4007fea2c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:55.677782 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1339/2675] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:53:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:55.682408 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f8ec2-7096-48cc-8b49-919252ee0f0a admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:55.744996 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f8ec2-7096-48cc-8b49-919252ee0f0a admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:55.749702 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db85c26-9716-4e02-ad8d-097a6712c9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:55.750377 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db85c26-9716-4e02-ad8d-097a6712c9cd admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:55.750641 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db85c26-9716-4e02-ad8d-097a6712c9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:55.750919 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db85c26-9716-4e02-ad8d-097a6712c9cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:55.751651 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db85c26-9716-4e02-ad8d-097a6712c9cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:55.753639 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db85c26-9716-4e02-ad8d-097a6712c9cd admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:55.779877 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1337/2676] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:53:55 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:53:55.783952 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c806321d-282f-444e-8fba-a4007fea2c3b None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:55.795064 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c806321d-282f-444e-8fba-a4007fea2c3b None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:55.828261 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:55.853772 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e990c-85d0-44e9-a5ea-4e7eaf3525fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:55.854440 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e990c-85d0-44e9-a5ea-4e7eaf3525fe admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:55.854724 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e990c-85d0-44e9-a5ea-4e7eaf3525fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:55.855016 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e990c-85d0-44e9-a5ea-4e7eaf3525fe admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:55.855716 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e990c-85d0-44e9-a5ea-4e7eaf3525fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:55.857759 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e990c-85d0-44e9-a5ea-4e7eaf3525fe admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:55.877854 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1340/2677] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:53:55 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:53:55.881737 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db85c26-9716-4e02-ad8d-097a6712c9cd admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:55.892568 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db85c26-9716-4e02-ad8d-097a6712c9cd admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:55.897168 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff71cf-75cc-4dd9-b2a8-b43c99c1bdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:55.897821 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff71cf-75cc-4dd9-b2a8-b43c99c1bdce admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:55.898107 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff71cf-75cc-4dd9-b2a8-b43c99c1bdce admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:55.898386 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff71cf-75cc-4dd9-b2a8-b43c99c1bdce admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:55.899154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff71cf-75cc-4dd9-b2a8-b43c99c1bdce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:55.901287 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff71cf-75cc-4dd9-b2a8-b43c99c1bdce admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:55.911876 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1338/2678] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:53:55.915968 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e990c-85d0-44e9-a5ea-4e7eaf3525fe admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:55.926210 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e990c-85d0-44e9-a5ea-4e7eaf3525fe admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:55.930860 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5695c23-47f0-48ef-97ee-db171d7e9e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:55.931543 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5695c23-47f0-48ef-97ee-db171d7e9e7c admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:55.931826 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5695c23-47f0-48ef-97ee-db171d7e9e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:55.932102 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5695c23-47f0-48ef-97ee-db171d7e9e7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:55.932857 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5695c23-47f0-48ef-97ee-db171d7e9e7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:55.934435 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5695c23-47f0-48ef-97ee-db171d7e9e7c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:55.944209 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1341/2679] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:53:55.948830 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff71cf-75cc-4dd9-b2a8-b43c99c1bdce admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:55.958576 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff71cf-75cc-4dd9-b2a8-b43c99c1bdce admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:55.963956 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ed1e8-1956-4dd0-a04a-18a74978fab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:55.964871 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ed1e8-1956-4dd0-a04a-18a74978fab7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:55.965159 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ed1e8-1956-4dd0-a04a-18a74978fab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:55.965525 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ed1e8-1956-4dd0-a04a-18a74978fab7 admin admin] PATH_INFO: `/v3/projects/7126ef2c63944fcd904c5783d99bf207/users/b19d6286b5f14cf6920dafe761c4726a/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:55.980255 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03ed1e8-1956-4dd0-a04a-18a74978fab7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7126ef2c63944fcd904c5783d99bf207, user_id=b19d6286b5f14cf6920dafe761c4726a, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:55.982980 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03ed1e8-1956-4dd0-a04a-18a74978fab7 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:55.996375 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1339/2680] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:53:55 2022] PUT /identity/v3/projects/7126ef2c63944fcd904c5783d99bf207/users/b19d6286b5f14cf6920dafe761c4726a/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:56.006481 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515f5d4-6d73-4d0c-a687-efa2a8b027ee None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:56.006714 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515f5d4-6d73-4d0c-a687-efa2a8b027ee None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:56.007001 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515f5d4-6d73-4d0c-a687-efa2a8b027ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:56.033473 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-c515f5d4-6d73-4d0c-a687-efa2a8b027ee None None] MFA Rules not processed for user `b19d6286b5f14cf6920dafe761c4726a`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:56.034679 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c515f5d4-6d73-4d0c-a687-efa2a8b027ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:56.098025 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1342/2681] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:53:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:56.102661 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ed1e8-1956-4dd0-a04a-18a74978fab7 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:56.146897 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:56.165007 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ed1e8-1956-4dd0-a04a-18a74978fab7 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:56.169652 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77192b5b-4dff-4023-a13f-bd3aa9ca34e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:56.170297 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77192b5b-4dff-4023-a13f-bd3aa9ca34e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:56.170578 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77192b5b-4dff-4023-a13f-bd3aa9ca34e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:56.170854 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77192b5b-4dff-4023-a13f-bd3aa9ca34e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:56.171584 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77192b5b-4dff-4023-a13f-bd3aa9ca34e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:56.173474 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77192b5b-4dff-4023-a13f-bd3aa9ca34e7 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:56.198745 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1340/2682] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:53:56 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:53:56.202941 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c515f5d4-6d73-4d0c-a687-efa2a8b027ee None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:56.213861 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c515f5d4-6d73-4d0c-a687-efa2a8b027ee None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:56.283747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d44786-2865-4b8b-b4d5-d00bf1e30977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:56.284338 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d44786-2865-4b8b-b4d5-d00bf1e30977 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:56.284626 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d44786-2865-4b8b-b4d5-d00bf1e30977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:56.284905 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d44786-2865-4b8b-b4d5-d00bf1e30977 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:56.285607 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d44786-2865-4b8b-b4d5-d00bf1e30977 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:56.287567 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d44786-2865-4b8b-b4d5-d00bf1e30977 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:56.307904 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1343/2683] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:53:56 2022] POST /identity/v3/users => generated 298 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:53:56.311854 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77192b5b-4dff-4023-a13f-bd3aa9ca34e7 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:56.322983 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77192b5b-4dff-4023-a13f-bd3aa9ca34e7 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:56.328017 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a8880d-d610-4643-8a7a-7b5092c200a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:56.328677 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8880d-d610-4643-8a7a-7b5092c200a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:56.328964 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8880d-d610-4643-8a7a-7b5092c200a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:56.329243 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8880d-d610-4643-8a7a-7b5092c200a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:56.329969 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a8880d-d610-4643-8a7a-7b5092c200a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:56.331608 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a8880d-d610-4643-8a7a-7b5092c200a6 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:56.342664 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1341/2684] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:53:56.346808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d44786-2865-4b8b-b4d5-d00bf1e30977 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:56.357346 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d44786-2865-4b8b-b4d5-d00bf1e30977 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:56.362102 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7dcdca-2d27-4331-90e2-86886dd99634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:56.362978 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7dcdca-2d27-4331-90e2-86886dd99634 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:56.363324 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7dcdca-2d27-4331-90e2-86886dd99634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:56.363599 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7dcdca-2d27-4331-90e2-86886dd99634 admin admin] PATH_INFO: `/v3/projects/e51bfb1c46a442c687027dc7c859a6d4/users/ea1b68041f6249888f8658bf25f33657/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:56.378902 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7dcdca-2d27-4331-90e2-86886dd99634 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e51bfb1c46a442c687027dc7c859a6d4, user_id=ea1b68041f6249888f8658bf25f33657, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:56.381577 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7dcdca-2d27-4331-90e2-86886dd99634 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:56.394743 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1344/2685] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:53:56 2022] PUT /identity/v3/projects/e51bfb1c46a442c687027dc7c859a6d4/users/ea1b68041f6249888f8658bf25f33657/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:56.398622 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a8880d-d610-4643-8a7a-7b5092c200a6 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:56.408681 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a8880d-d610-4643-8a7a-7b5092c200a6 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:56.466018 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c499f-88b5-47a2-869c-d01a42dad4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:56.466680 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c499f-88b5-47a2-869c-d01a42dad4ef admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:56.466963 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c499f-88b5-47a2-869c-d01a42dad4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:56.467277 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c499f-88b5-47a2-869c-d01a42dad4ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:56.467995 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c499f-88b5-47a2-869c-d01a42dad4ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:56.469587 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c499f-88b5-47a2-869c-d01a42dad4ef admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:56.480059 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1342/2686] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:56 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:53:56.484184 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7dcdca-2d27-4331-90e2-86886dd99634 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:56.494851 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7dcdca-2d27-4331-90e2-86886dd99634 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:56.503424 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e96a47-136b-4351-bb40-0122145b6469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:56.503424 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e96a47-136b-4351-bb40-0122145b6469 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:56.503424 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e96a47-136b-4351-bb40-0122145b6469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:56.503424 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e96a47-136b-4351-bb40-0122145b6469 admin admin] PATH_INFO: `/v3/projects/e51bfb1c46a442c687027dc7c859a6d4/users/ea1b68041f6249888f8658bf25f33657/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:56.506011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e96a47-136b-4351-bb40-0122145b6469 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e51bfb1c46a442c687027dc7c859a6d4, user_id=ea1b68041f6249888f8658bf25f33657, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:56.506011 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e96a47-136b-4351-bb40-0122145b6469 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:56.508733 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:56.519493 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1345/2687] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:53:56 2022] PUT /identity/v3/projects/e51bfb1c46a442c687027dc7c859a6d4/users/ea1b68041f6249888f8658bf25f33657/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:56.529781 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67dc7df-016b-4fcf-9819-cba8badbc2b1 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:56.530079 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67dc7df-016b-4fcf-9819-cba8badbc2b1 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:56.530367 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67dc7df-016b-4fcf-9819-cba8badbc2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:56.558567 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-a67dc7df-016b-4fcf-9819-cba8badbc2b1 None None] MFA Rules not processed for user `ea1b68041f6249888f8658bf25f33657`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:56.559794 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-a67dc7df-016b-4fcf-9819-cba8badbc2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:56.631112 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1343/2688] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:53:56 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:56.643423 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e96a47-136b-4351-bb40-0122145b6469 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:56.645206 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-29e96a47-136b-4351-bb40-0122145b6469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:56.703516 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e96a47-136b-4351-bb40-0122145b6469 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:56.707224 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12bfe46-3b26-49dc-a380-9bb23b6b604c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:56.707876 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bfe46-3b26-49dc-a380-9bb23b6b604c service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:56.708184 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bfe46-3b26-49dc-a380-9bb23b6b604c service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:56.708441 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bfe46-3b26-49dc-a380-9bb23b6b604c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:56.763697 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12bfe46-3b26-49dc-a380-9bb23b6b604c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:56.765139 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12bfe46-3b26-49dc-a380-9bb23b6b604c service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:56.774019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1346/2689] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:58.274413 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67dc7df-016b-4fcf-9819-cba8badbc2b1 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:58.276411 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-a67dc7df-016b-4fcf-9819-cba8badbc2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:58.288762 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67dc7df-016b-4fcf-9819-cba8badbc2b1 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:58.293548 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd40b32-7f63-404b-a646-d9128d37fc92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:58.294188 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd40b32-7f63-404b-a646-d9128d37fc92 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:58.294470 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd40b32-7f63-404b-a646-d9128d37fc92 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:58.294746 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd40b32-7f63-404b-a646-d9128d37fc92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:58.331019 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:58.356915 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd40b32-7f63-404b-a646-d9128d37fc92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:58.358432 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd40b32-7f63-404b-a646-d9128d37fc92 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:58.368351 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1344/2690] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:58.907046 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12bfe46-3b26-49dc-a380-9bb23b6b604c service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:58.916438 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12bfe46-3b26-49dc-a380-9bb23b6b604c service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:58.920207 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a71657-83f1-438e-b4ba-8002ff592015 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:58.921110 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a71657-83f1-438e-b4ba-8002ff592015 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:58.921464 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a71657-83f1-438e-b4ba-8002ff592015 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:58.921810 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a71657-83f1-438e-b4ba-8002ff592015 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:58.931645 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a71657-83f1-438e-b4ba-8002ff592015 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:58.933198 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a71657-83f1-438e-b4ba-8002ff592015 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:58.943310 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1347/2691] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:58 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:54:05.926028 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd40b32-7f63-404b-a646-d9128d37fc92 service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:05.987390 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd40b32-7f63-404b-a646-d9128d37fc92 service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:05.991872 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68585a64-3b53-43c1-88d6-ae8760eac077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:05.992738 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68585a64-3b53-43c1-88d6-ae8760eac077 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:05.993023 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68585a64-3b53-43c1-88d6-ae8760eac077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:05.993301 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68585a64-3b53-43c1-88d6-ae8760eac077 admin admin] PATH_INFO: `/v3/users/ec669b302f9b4067b587bea4b75fd918` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:05.994499 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68585a64-3b53-43c1-88d6-ae8760eac077 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec669b302f9b4067b587bea4b75fd918)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:05.996448 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68585a64-3b53-43c1-88d6-ae8760eac077 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:06.043309 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-68585a64-3b53-43c1-88d6-ae8760eac077 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec669b302f9b4067b587bea4b75fd918'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:06.043974 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-68585a64-3b53-43c1-88d6-ae8760eac077 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec669b302f9b4067b587bea4b75fd918'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:06.057945 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-68585a64-3b53-43c1-88d6-ae8760eac077 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec669b302f9b4067b587bea4b75fd918'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:06.065935 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-68585a64-3b53-43c1-88d6-ae8760eac077 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec669b302f9b4067b587bea4b75fd918'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:06.073716 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-68585a64-3b53-43c1-88d6-ae8760eac077 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec669b302f9b4067b587bea4b75fd918'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:06.079646 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1345/2692] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:54:05 2022] DELETE /identity/v3/users/ec669b302f9b4067b587bea4b75fd918 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:54:06.084085 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c86109-0d87-4585-9913-bf56509298ff None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:06.084550 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c86109-0d87-4585-9913-bf56509298ff None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:06.084645 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c86109-0d87-4585-9913-bf56509298ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:06.102070 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-28c86109-0d87-4585-9913-bf56509298ff None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:54:06.103294 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-28c86109-0d87-4585-9913-bf56509298ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:06.161092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1348/2693] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:54:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:54:06.171204 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68585a64-3b53-43c1-88d6-ae8760eac077 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:06.172586 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-68585a64-3b53-43c1-88d6-ae8760eac077 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:06.228477 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68585a64-3b53-43c1-88d6-ae8760eac077 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:06.232140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d38baab-8c09-43a9-a4c7-d0938c85f58c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:06.232716 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d38baab-8c09-43a9-a4c7-d0938c85f58c service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:06.232993 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d38baab-8c09-43a9-a4c7-d0938c85f58c service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:06.233264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d38baab-8c09-43a9-a4c7-d0938c85f58c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:06.242083 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d38baab-8c09-43a9-a4c7-d0938c85f58c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:06.243755 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d38baab-8c09-43a9-a4c7-d0938c85f58c service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:06.247654 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:54:06.255439 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1346/2694] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:54:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:54:06.524682 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c86109-0d87-4585-9913-bf56509298ff None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:06.526193 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-28c86109-0d87-4585-9913-bf56509298ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:06.538543 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c86109-0d87-4585-9913-bf56509298ff None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:06.543340 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4aa2d2-5afd-4502-9436-f50036fe8787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:06.544613 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4aa2d2-5afd-4502-9436-f50036fe8787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:06.545029 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4aa2d2-5afd-4502-9436-f50036fe8787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:06.545318 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4aa2d2-5afd-4502-9436-f50036fe8787 admin admin] PATH_INFO: `/v3/projects/f00730bc0c964ca082ce22d66b613847` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:06.546547 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4aa2d2-5afd-4502-9436-f50036fe8787 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f00730bc0c964ca082ce22d66b613847)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:06.548505 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4aa2d2-5afd-4502-9436-f50036fe8787 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:06.584659 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:54:06.597407 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-8e4aa2d2-5afd-4502-9436-f50036fe8787 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f00730bc0c964ca082ce22d66b613847', 'request_id': 'req-8e4aa2d2-5afd-4502-9436-f50036fe8787'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:06.606975 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-8e4aa2d2-5afd-4502-9436-f50036fe8787 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f00730bc0c964ca082ce22d66b613847', 'request_id': 'req-8e4aa2d2-5afd-4502-9436-f50036fe8787'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:06.614096 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-8e4aa2d2-5afd-4502-9436-f50036fe8787 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f00730bc0c964ca082ce22d66b613847', 'request_id': 'req-8e4aa2d2-5afd-4502-9436-f50036fe8787'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:06.625205 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-8e4aa2d2-5afd-4502-9436-f50036fe8787 admin admin] The token cache is being invalidate because project f00730bc0c964ca082ce22d66b613847 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:54:06.625489 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-8e4aa2d2-5afd-4502-9436-f50036fe8787 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:06.626153 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-8e4aa2d2-5afd-4502-9436-f50036fe8787 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:06.627767 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1349/2695] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:54:06 2022] DELETE /identity/v3/projects/f00730bc0c964ca082ce22d66b613847 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:54:06.631608 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d38baab-8c09-43a9-a4c7-d0938c85f58c service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:06.641760 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d38baab-8c09-43a9-a4c7-d0938c85f58c service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:06.698560 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297fc10f-aa5c-4a59-88ee-0d2eed488c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:06.699451 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297fc10f-aa5c-4a59-88ee-0d2eed488c22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:06.699735 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297fc10f-aa5c-4a59-88ee-0d2eed488c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:06.700009 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297fc10f-aa5c-4a59-88ee-0d2eed488c22 admin admin] PATH_INFO: `/v3/users/b19d6286b5f14cf6920dafe761c4726a` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:06.701179 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297fc10f-aa5c-4a59-88ee-0d2eed488c22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b19d6286b5f14cf6920dafe761c4726a)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:06.703161 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297fc10f-aa5c-4a59-88ee-0d2eed488c22 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:06.751379 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-297fc10f-aa5c-4a59-88ee-0d2eed488c22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b19d6286b5f14cf6920dafe761c4726a'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:06.752030 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-297fc10f-aa5c-4a59-88ee-0d2eed488c22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b19d6286b5f14cf6920dafe761c4726a'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:06.764276 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-297fc10f-aa5c-4a59-88ee-0d2eed488c22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b19d6286b5f14cf6920dafe761c4726a'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:06.771533 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-297fc10f-aa5c-4a59-88ee-0d2eed488c22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b19d6286b5f14cf6920dafe761c4726a'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:06.778327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-297fc10f-aa5c-4a59-88ee-0d2eed488c22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b19d6286b5f14cf6920dafe761c4726a'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:06.784207 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1347/2696] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:54:06 2022] DELETE /identity/v3/users/b19d6286b5f14cf6920dafe761c4726a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:54:07.057076 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4aa2d2-5afd-4502-9436-f50036fe8787 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:07.058640 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-8e4aa2d2-5afd-4502-9436-f50036fe8787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:07.072438 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4aa2d2-5afd-4502-9436-f50036fe8787 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:07.129699 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182845b5-9c6d-4baa-8344-5013fb2f689c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:07.130585 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182845b5-9c6d-4baa-8344-5013fb2f689c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:07.130851 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182845b5-9c6d-4baa-8344-5013fb2f689c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:07.131173 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182845b5-9c6d-4baa-8344-5013fb2f689c admin admin] PATH_INFO: `/v3/projects/7126ef2c63944fcd904c5783d99bf207` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:07.132360 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182845b5-9c6d-4baa-8344-5013fb2f689c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7126ef2c63944fcd904c5783d99bf207)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:07.134784 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182845b5-9c6d-4baa-8344-5013fb2f689c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:07.179676 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-182845b5-9c6d-4baa-8344-5013fb2f689c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7126ef2c63944fcd904c5783d99bf207', 'request_id': 'req-182845b5-9c6d-4baa-8344-5013fb2f689c'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:07.187734 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-182845b5-9c6d-4baa-8344-5013fb2f689c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7126ef2c63944fcd904c5783d99bf207', 'request_id': 'req-182845b5-9c6d-4baa-8344-5013fb2f689c'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:07.194515 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-182845b5-9c6d-4baa-8344-5013fb2f689c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7126ef2c63944fcd904c5783d99bf207', 'request_id': 'req-182845b5-9c6d-4baa-8344-5013fb2f689c'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:07.205189 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-182845b5-9c6d-4baa-8344-5013fb2f689c admin admin] The token cache is being invalidate because project 7126ef2c63944fcd904c5783d99bf207 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:54:07.205474 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-182845b5-9c6d-4baa-8344-5013fb2f689c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:07.206114 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-182845b5-9c6d-4baa-8344-5013fb2f689c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:07.207680 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1350/2697] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:54:07 2022] DELETE /identity/v3/projects/7126ef2c63944fcd904c5783d99bf207 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:54:07.211768 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297fc10f-aa5c-4a59-88ee-0d2eed488c22 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:07.213778 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-297fc10f-aa5c-4a59-88ee-0d2eed488c22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:07.226380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297fc10f-aa5c-4a59-88ee-0d2eed488c22 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:07.282773 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3b0369-ebf0-4d96-ae16-93e032808016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:07.283683 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b0369-ebf0-4d96-ae16-93e032808016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:07.283966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b0369-ebf0-4d96-ae16-93e032808016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:07.284241 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b0369-ebf0-4d96-ae16-93e032808016 admin admin] PATH_INFO: `/v3/users/ea1b68041f6249888f8658bf25f33657` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:07.285453 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3b0369-ebf0-4d96-ae16-93e032808016 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea1b68041f6249888f8658bf25f33657)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:07.287434 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3b0369-ebf0-4d96-ae16-93e032808016 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:07.308872 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:54:07.339469 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ba3b0369-ebf0-4d96-ae16-93e032808016 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea1b68041f6249888f8658bf25f33657'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:07.340127 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ba3b0369-ebf0-4d96-ae16-93e032808016 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea1b68041f6249888f8658bf25f33657'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:07.351892 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ba3b0369-ebf0-4d96-ae16-93e032808016 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea1b68041f6249888f8658bf25f33657'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:07.359101 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ba3b0369-ebf0-4d96-ae16-93e032808016 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea1b68041f6249888f8658bf25f33657'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:07.365979 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ba3b0369-ebf0-4d96-ae16-93e032808016 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea1b68041f6249888f8658bf25f33657'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:07.372767 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1348/2698] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:54:07 2022] DELETE /identity/v3/users/ea1b68041f6249888f8658bf25f33657 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:54:07.642222 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182845b5-9c6d-4baa-8344-5013fb2f689c admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:07.644096 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-182845b5-9c6d-4baa-8344-5013fb2f689c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:07.657170 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182845b5-9c6d-4baa-8344-5013fb2f689c admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:07.704895 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:54:07.716986 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979191d-bba3-40d7-a944-c0b681218cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:07.717845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979191d-bba3-40d7-a944-c0b681218cbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:07.718129 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979191d-bba3-40d7-a944-c0b681218cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:07.718519 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979191d-bba3-40d7-a944-c0b681218cbb admin admin] PATH_INFO: `/v3/projects/e51bfb1c46a442c687027dc7c859a6d4` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:07.719721 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5979191d-bba3-40d7-a944-c0b681218cbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e51bfb1c46a442c687027dc7c859a6d4)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:07.721666 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5979191d-bba3-40d7-a944-c0b681218cbb admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:07.776344 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-5979191d-bba3-40d7-a944-c0b681218cbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e51bfb1c46a442c687027dc7c859a6d4', 'request_id': 'req-5979191d-bba3-40d7-a944-c0b681218cbb'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:07.783588 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-5979191d-bba3-40d7-a944-c0b681218cbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e51bfb1c46a442c687027dc7c859a6d4', 'request_id': 'req-5979191d-bba3-40d7-a944-c0b681218cbb'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:07.790676 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-5979191d-bba3-40d7-a944-c0b681218cbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e51bfb1c46a442c687027dc7c859a6d4', 'request_id': 'req-5979191d-bba3-40d7-a944-c0b681218cbb'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:07.800766 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-5979191d-bba3-40d7-a944-c0b681218cbb admin admin] The token cache is being invalidate because project e51bfb1c46a442c687027dc7c859a6d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:54:07.801052 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-5979191d-bba3-40d7-a944-c0b681218cbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:07.801686 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-5979191d-bba3-40d7-a944-c0b681218cbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:07.803262 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1351/2699] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:54:07 2022] DELETE /identity/v3/projects/e51bfb1c46a442c687027dc7c859a6d4 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:54:07.810432 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04319f1-ee9b-49c6-a4d3-2a1f8ed729a6 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:07.810728 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04319f1-ee9b-49c6-a4d3-2a1f8ed729a6 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:07.811032 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04319f1-ee9b-49c6-a4d3-2a1f8ed729a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:07.830117 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-c04319f1-ee9b-49c6-a4d3-2a1f8ed729a6 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:54:07.831348 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c04319f1-ee9b-49c6-a4d3-2a1f8ed729a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:07.889311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1349/2700] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:54:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:54:07.897199 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12ce11-75f2-4fdf-8197-6e10e01c4ea1 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:07.897487 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12ce11-75f2-4fdf-8197-6e10e01c4ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:07.897766 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12ce11-75f2-4fdf-8197-6e10e01c4ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:07.916727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-fb12ce11-75f2-4fdf-8197-6e10e01c4ea1 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:54:07.917911 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-fb12ce11-75f2-4fdf-8197-6e10e01c4ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:07.926162 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1352/2701] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:54:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:54:07.930349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04319f1-ee9b-49c6-a4d3-2a1f8ed729a6 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:07.940948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04319f1-ee9b-49c6-a4d3-2a1f8ed729a6 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:07.945543 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44105ad1-98ec-437f-9703-26f516a07766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:07.946242 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44105ad1-98ec-437f-9703-26f516a07766 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:07.946525 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44105ad1-98ec-437f-9703-26f516a07766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:07.946801 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44105ad1-98ec-437f-9703-26f516a07766 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:07.947550 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44105ad1-98ec-437f-9703-26f516a07766 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:54:07.947855 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44105ad1-98ec-437f-9703-26f516a07766 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:07.949450 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44105ad1-98ec-437f-9703-26f516a07766 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:07.960870 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1350/2702] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:54:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:54:07.965606 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb12ce11-75f2-4fdf-8197-6e10e01c4ea1 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:07.974903 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb12ce11-75f2-4fdf-8197-6e10e01c4ea1 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:07.980231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b82472-0f50-4ae9-8966-0b9b7592d78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:07.980887 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b82472-0f50-4ae9-8966-0b9b7592d78d admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:07.981151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b82472-0f50-4ae9-8966-0b9b7592d78d admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:07.981427 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b82472-0f50-4ae9-8966-0b9b7592d78d admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:07.982119 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b82472-0f50-4ae9-8966-0b9b7592d78d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:07.984140 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b82472-0f50-4ae9-8966-0b9b7592d78d admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:08.009068 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1353/2703] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:54:07 2022] POST /identity/v3/projects => generated 379 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:54:08.013185 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44105ad1-98ec-437f-9703-26f516a07766 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:08.023106 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44105ad1-98ec-437f-9703-26f516a07766 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:08.078899 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2180d1-c39d-4834-b197-a303fb841eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:08.079592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2180d1-c39d-4834-b197-a303fb841eee admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:08.079877 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2180d1-c39d-4834-b197-a303fb841eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:08.080152 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2180d1-c39d-4834-b197-a303fb841eee admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:08.080846 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2180d1-c39d-4834-b197-a303fb841eee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:08.082764 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2180d1-c39d-4834-b197-a303fb841eee admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:08.102313 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1351/2704] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:54:08 2022] POST /identity/v3/users => generated 290 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:54:08.107191 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b82472-0f50-4ae9-8966-0b9b7592d78d admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:08.117294 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b82472-0f50-4ae9-8966-0b9b7592d78d admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:08.122025 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7137f-9f00-44e0-95e9-11b9ad9eb5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:08.122699 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7137f-9f00-44e0-95e9-11b9ad9eb5b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:08.122900 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7137f-9f00-44e0-95e9-11b9ad9eb5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:08.123204 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7137f-9f00-44e0-95e9-11b9ad9eb5b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:08.123934 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f7137f-9f00-44e0-95e9-11b9ad9eb5b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:08.125521 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f7137f-9f00-44e0-95e9-11b9ad9eb5b6 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:08.135747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1354/2705] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:54:08.139627 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2180d1-c39d-4834-b197-a303fb841eee admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:08.149939 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2180d1-c39d-4834-b197-a303fb841eee admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:08.154473 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d879f6-b95a-43d2-853e-aafc3a331377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:08.155163 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d879f6-b95a-43d2-853e-aafc3a331377 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:08.155449 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d879f6-b95a-43d2-853e-aafc3a331377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:08.155724 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d879f6-b95a-43d2-853e-aafc3a331377 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:08.156429 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d879f6-b95a-43d2-853e-aafc3a331377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:08.158002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d879f6-b95a-43d2-853e-aafc3a331377 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:08.168544 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1352/2706] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:54:08.172509 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7137f-9f00-44e0-95e9-11b9ad9eb5b6 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:08.183061 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7137f-9f00-44e0-95e9-11b9ad9eb5b6 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:08.187948 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5e18d-27d6-402a-a21e-4a6b5728c0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:08.188859 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5e18d-27d6-402a-a21e-4a6b5728c0bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:08.189138 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5e18d-27d6-402a-a21e-4a6b5728c0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:08.189410 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5e18d-27d6-402a-a21e-4a6b5728c0bd admin admin] PATH_INFO: `/v3/projects/59120b6715c1485db0cb1277a52e6c55/users/1b23fa0b7fdb4016917987ea1c40af33/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:08.203614 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c5e18d-27d6-402a-a21e-4a6b5728c0bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=59120b6715c1485db0cb1277a52e6c55, user_id=1b23fa0b7fdb4016917987ea1c40af33, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:08.206235 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c5e18d-27d6-402a-a21e-4a6b5728c0bd admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:08.220010 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1355/2707] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:54:08 2022] PUT /identity/v3/projects/59120b6715c1485db0cb1277a52e6c55/users/1b23fa0b7fdb4016917987ea1c40af33/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:54:08.225261 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3756b-01fb-446e-a3e8-e3abc59610ef None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:08.225726 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3756b-01fb-446e-a3e8-e3abc59610ef None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:08.225823 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3756b-01fb-446e-a3e8-e3abc59610ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:08.232319 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:54:08.246864 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-85e3756b-01fb-446e-a3e8-e3abc59610ef None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:54:08.248099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-85e3756b-01fb-446e-a3e8-e3abc59610ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:08.306807 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1353/2708] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:54:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:54:08.316226 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5e18d-27d6-402a-a21e-4a6b5728c0bd admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:08.317899 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-91c5e18d-27d6-402a-a21e-4a6b5728c0bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:08.383016 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5e18d-27d6-402a-a21e-4a6b5728c0bd admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:08.387139 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425eb396-fd4a-411d-9ea2-65e27d03ea91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:08.387793 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425eb396-fd4a-411d-9ea2-65e27d03ea91 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:08.388077 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425eb396-fd4a-411d-9ea2-65e27d03ea91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:08.388353 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425eb396-fd4a-411d-9ea2-65e27d03ea91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:08.397254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425eb396-fd4a-411d-9ea2-65e27d03ea91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:08.398693 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425eb396-fd4a-411d-9ea2-65e27d03ea91 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:08.408587 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1356/2709] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:54:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:54:10.746203 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13498189-c7b7-40ff-8b66-a765612d133c None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:10.747259 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13498189-c7b7-40ff-8b66-a765612d133c None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:10.747722 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13498189-c7b7-40ff-8b66-a765612d133c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:10.769489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-13498189-c7b7-40ff-8b66-a765612d133c None None] MFA Rules not processed for user `235cd365a5ab41cda9bffd0cee150617`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:54:10.770741 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-13498189-c7b7-40ff-8b66-a765612d133c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:10.849452 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1354/2710] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:54:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:54:12.729264 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e478f-bae7-44fd-9e3c-a966076eda4d None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:12.729264 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e478f-bae7-44fd-9e3c-a966076eda4d None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:12.729738 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e478f-bae7-44fd-9e3c-a966076eda4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:12.759803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-582e478f-bae7-44fd-9e3c-a966076eda4d None None] MFA Rules not processed for user `1b23fa0b7fdb4016917987ea1c40af33`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:54:12.761400 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-582e478f-bae7-44fd-9e3c-a966076eda4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:12.773067 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:54:12.867292 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1357/2711] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:54:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:54:12.872681 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13498189-c7b7-40ff-8b66-a765612d133c None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:12.885360 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13498189-c7b7-40ff-8b66-a765612d133c None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:12.890818 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144903a-a10a-4898-98c2-6c1d037c4577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:12.891556 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144903a-a10a-4898-98c2-6c1d037c4577 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:12.891844 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144903a-a10a-4898-98c2-6c1d037c4577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:12.892144 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144903a-a10a-4898-98c2-6c1d037c4577 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:12.892849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1144903a-a10a-4898-98c2-6c1d037c4577 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:12.894798 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1144903a-a10a-4898-98c2-6c1d037c4577 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:12.934188 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1355/2712] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:54:12 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:54:12.935644 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e478f-bae7-44fd-9e3c-a966076eda4d None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:12.948676 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e478f-bae7-44fd-9e3c-a966076eda4d None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:13.029066 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7c859-901e-4eff-b974-5422f29e7351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:13.029779 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7c859-901e-4eff-b974-5422f29e7351 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:13.030208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7c859-901e-4eff-b974-5422f29e7351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:13.030519 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7c859-901e-4eff-b974-5422f29e7351 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:13.032152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f7c859-901e-4eff-b974-5422f29e7351 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:13.035426 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f7c859-901e-4eff-b974-5422f29e7351 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:13.058123 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1358/2713] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:54:12 2022] POST /identity/v3/users => generated 296 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:54:13.061924 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144903a-a10a-4898-98c2-6c1d037c4577 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:13.073400 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144903a-a10a-4898-98c2-6c1d037c4577 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:13.078461 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13246e2c-d27c-4433-96fd-9a8b34b5b682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:13.079170 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13246e2c-d27c-4433-96fd-9a8b34b5b682 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:13.079457 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13246e2c-d27c-4433-96fd-9a8b34b5b682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:13.079735 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13246e2c-d27c-4433-96fd-9a8b34b5b682 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:13.080470 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13246e2c-d27c-4433-96fd-9a8b34b5b682 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:13.082106 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13246e2c-d27c-4433-96fd-9a8b34b5b682 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:13.096126 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1356/2714] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:54:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:54:13.100982 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7c859-901e-4eff-b974-5422f29e7351 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:13.114920 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7c859-901e-4eff-b974-5422f29e7351 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:13.122912 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc1d2e-d8de-417e-a5ba-f2f7a96b344c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:13.123851 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc1d2e-d8de-417e-a5ba-f2f7a96b344c admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:13.124140 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc1d2e-d8de-417e-a5ba-f2f7a96b344c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:13.124428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc1d2e-d8de-417e-a5ba-f2f7a96b344c admin admin] PATH_INFO: `/v3/projects/2ac479c9f83943b0801b60719412e9c4/users/ff97fc23806243179bd7b1d18d1ec811/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:13.140377 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cc1d2e-d8de-417e-a5ba-f2f7a96b344c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ac479c9f83943b0801b60719412e9c4, user_id=ff97fc23806243179bd7b1d18d1ec811, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:13.143282 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cc1d2e-d8de-417e-a5ba-f2f7a96b344c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:13.157938 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1359/2715] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:54:13 2022] PUT /identity/v3/projects/2ac479c9f83943b0801b60719412e9c4/users/ff97fc23806243179bd7b1d18d1ec811/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:54:13.162004 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13246e2c-d27c-4433-96fd-9a8b34b5b682 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:13.172423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13246e2c-d27c-4433-96fd-9a8b34b5b682 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:13.236172 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f51e814-89f6-4c3a-a998-63315291da61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:13.237212 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51e814-89f6-4c3a-a998-63315291da61 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:13.237587 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51e814-89f6-4c3a-a998-63315291da61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:13.237822 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51e814-89f6-4c3a-a998-63315291da61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:13.238562 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f51e814-89f6-4c3a-a998-63315291da61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:13.240250 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f51e814-89f6-4c3a-a998-63315291da61 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:13.253050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1357/2716] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:54:13 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:54:13.257124 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc1d2e-d8de-417e-a5ba-f2f7a96b344c admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:13.270472 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc1d2e-d8de-417e-a5ba-f2f7a96b344c admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:13.275538 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd81ffe-0536-4abb-bff2-9dde016d5b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:13.276579 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd81ffe-0536-4abb-bff2-9dde016d5b92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:13.276865 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd81ffe-0536-4abb-bff2-9dde016d5b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:13.277143 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd81ffe-0536-4abb-bff2-9dde016d5b92 admin admin] PATH_INFO: `/v3/projects/2ac479c9f83943b0801b60719412e9c4/users/ff97fc23806243179bd7b1d18d1ec811/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:13.279465 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd81ffe-0536-4abb-bff2-9dde016d5b92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ac479c9f83943b0801b60719412e9c4, user_id=ff97fc23806243179bd7b1d18d1ec811, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:13.282254 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd81ffe-0536-4abb-bff2-9dde016d5b92 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:13.294457 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1360/2717] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:54:13 2022] PUT /identity/v3/projects/2ac479c9f83943b0801b60719412e9c4/users/ff97fc23806243179bd7b1d18d1ec811/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:54:18.279802 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c34ff-1ce8-4d75-b89c-1e0ea7cde476 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:18.280298 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c34ff-1ce8-4d75-b89c-1e0ea7cde476 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:18.280396 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c34ff-1ce8-4d75-b89c-1e0ea7cde476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:18.302190 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:54:18.313767 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-b06c34ff-1ce8-4d75-b89c-1e0ea7cde476 None None] MFA Rules not processed for user `ff97fc23806243179bd7b1d18d1ec811`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:54:18.315798 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b06c34ff-1ce8-4d75-b89c-1e0ea7cde476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:18.403169 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1358/2718] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:54:18 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:54:18.411398 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd81ffe-0536-4abb-bff2-9dde016d5b92 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:18.415813 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-bbd81ffe-0536-4abb-bff2-9dde016d5b92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:18.490972 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:54:18.513008 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd81ffe-0536-4abb-bff2-9dde016d5b92 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:18.520517 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded7f563-5cbd-48c7-9c6a-3f28fff670fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:18.521277 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7f563-5cbd-48c7-9c6a-3f28fff670fd service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:18.521620 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7f563-5cbd-48c7-9c6a-3f28fff670fd service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:18.521957 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7f563-5cbd-48c7-9c6a-3f28fff670fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:18.596185 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded7f563-5cbd-48c7-9c6a-3f28fff670fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:18.596185 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded7f563-5cbd-48c7-9c6a-3f28fff670fd service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:18.604102 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1361/2719] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:54:18 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:54:28.393883 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=29 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:54:28.599986 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=28 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:54:40.164043 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c34ff-1ce8-4d75-b89c-1e0ea7cde476 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:40.165498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b06c34ff-1ce8-4d75-b89c-1e0ea7cde476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:40.222733 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c34ff-1ce8-4d75-b89c-1e0ea7cde476 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:40.226265 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27472b60-0e90-48c6-813b-d82b94b6b7e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:40.226934 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27472b60-0e90-48c6-813b-d82b94b6b7e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:40.227279 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27472b60-0e90-48c6-813b-d82b94b6b7e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:40.227563 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27472b60-0e90-48c6-813b-d82b94b6b7e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:40.229968 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-27472b60-0e90-48c6-813b-d82b94b6b7e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:40.313367 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27472b60-0e90-48c6-813b-d82b94b6b7e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:40.314803 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27472b60-0e90-48c6-813b-d82b94b6b7e8 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:40.333029 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1359/2720] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:54:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:54:48.248937 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded7f563-5cbd-48c7-9c6a-3f28fff670fd service nova] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:48.250578 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ded7f563-5cbd-48c7-9c6a-3f28fff670fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:48.341776 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded7f563-5cbd-48c7-9c6a-3f28fff670fd service nova] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:48.348155 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9446c75c-671a-4fa6-9d2d-276a00e858a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:48.348942 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446c75c-671a-4fa6-9d2d-276a00e858a9 service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:48.349352 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446c75c-671a-4fa6-9d2d-276a00e858a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:48.349731 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446c75c-671a-4fa6-9d2d-276a00e858a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:48.351677 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-9446c75c-671a-4fa6-9d2d-276a00e858a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:48.384229 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9446c75c-671a-4fa6-9d2d-276a00e858a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:48.387564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9446c75c-671a-4fa6-9d2d-276a00e858a9 service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:48.416630 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1362/2721] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:54:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:54:50.330469 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=61 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:54:52.969019 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27472b60-0e90-48c6-813b-d82b94b6b7e8 service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:53.041407 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27472b60-0e90-48c6-813b-d82b94b6b7e8 service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:53.045051 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5807f400-982e-497a-b303-ea4a0508e11b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:53.045688 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807f400-982e-497a-b303-ea4a0508e11b service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:53.045973 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807f400-982e-497a-b303-ea4a0508e11b service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:53.046248 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807f400-982e-497a-b303-ea4a0508e11b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:53.048046 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-5807f400-982e-497a-b303-ea4a0508e11b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:53.129443 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5807f400-982e-497a-b303-ea4a0508e11b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:53.130799 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5807f400-982e-497a-b303-ea4a0508e11b service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:53.149754 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1360/2722] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:54:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:54:55.885225 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03739251-e496-4951-aa3a-3df073bdcae8 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:55.885791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03739251-e496-4951-aa3a-3df073bdcae8 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:55.885791 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03739251-e496-4951-aa3a-3df073bdcae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:55.904804 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-03739251-e496-4951-aa3a-3df073bdcae8 None None] MFA Rules not processed for user `1b23fa0b7fdb4016917987ea1c40af33`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:54:55.906076 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-03739251-e496-4951-aa3a-3df073bdcae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:55.914428 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1363/2723] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:54:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:54:56.305716 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5807f400-982e-497a-b303-ea4a0508e11b service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:56.308380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-5807f400-982e-497a-b303-ea4a0508e11b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:56.321664 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5807f400-982e-497a-b303-ea4a0508e11b service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:56.327571 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6827a9-0461-46d0-aa90-6bed8a063599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:56.328665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6827a9-0461-46d0-aa90-6bed8a063599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:56.329026 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6827a9-0461-46d0-aa90-6bed8a063599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:56.329381 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6827a9-0461-46d0-aa90-6bed8a063599 admin admin] PATH_INFO: `/v3/users/25ba370b8daf49e48514cd49c2851215` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:56.330710 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6827a9-0461-46d0-aa90-6bed8a063599 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25ba370b8daf49e48514cd49c2851215)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:56.333848 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6827a9-0461-46d0-aa90-6bed8a063599 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:56.399580 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-5e6827a9-0461-46d0-aa90-6bed8a063599 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25ba370b8daf49e48514cd49c2851215'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:56.400285 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-5e6827a9-0461-46d0-aa90-6bed8a063599 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25ba370b8daf49e48514cd49c2851215'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:56.407730 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:54:56.413118 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03739251-e496-4951-aa3a-3df073bdcae8 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:56.414406 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-5e6827a9-0461-46d0-aa90-6bed8a063599 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25ba370b8daf49e48514cd49c2851215'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:56.415277 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-03739251-e496-4951-aa3a-3df073bdcae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:56.424080 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-5e6827a9-0461-46d0-aa90-6bed8a063599 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25ba370b8daf49e48514cd49c2851215'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:56.433423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-5e6827a9-0461-46d0-aa90-6bed8a063599 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25ba370b8daf49e48514cd49c2851215'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:56.441394 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1361/2724] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:54:56 2022] DELETE /identity/v3/users/25ba370b8daf49e48514cd49c2851215 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:54:56.445960 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03739251-e496-4951-aa3a-3df073bdcae8 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:56.503532 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66691f0d-cd49-4ff6-9033-b6f1f2924e70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:56.504196 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66691f0d-cd49-4ff6-9033-b6f1f2924e70 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:56.504483 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66691f0d-cd49-4ff6-9033-b6f1f2924e70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:56.504762 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66691f0d-cd49-4ff6-9033-b6f1f2924e70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:56.507308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-66691f0d-cd49-4ff6-9033-b6f1f2924e70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:56.518198 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66691f0d-cd49-4ff6-9033-b6f1f2924e70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:56.519681 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66691f0d-cd49-4ff6-9033-b6f1f2924e70 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:56.529306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1364/2725] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:54:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:54:56.734846 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6827a9-0461-46d0-aa90-6bed8a063599 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:56.737101 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-5e6827a9-0461-46d0-aa90-6bed8a063599 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:56.836411 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6827a9-0461-46d0-aa90-6bed8a063599 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:56.843556 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a6080f-2c1c-48cc-acaa-501f4ddd558d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:56.844504 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6080f-2c1c-48cc-acaa-501f4ddd558d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:56.844794 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6080f-2c1c-48cc-acaa-501f4ddd558d admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:56.845193 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6080f-2c1c-48cc-acaa-501f4ddd558d admin admin] PATH_INFO: `/v3/projects/54d57ba7dd0d441b9748fc600bc34972` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:56.847422 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a6080f-2c1c-48cc-acaa-501f4ddd558d admin admin] RBAC: Authorizing `identity:delete_project(project_id=54d57ba7dd0d441b9748fc600bc34972)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:56.849508 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a6080f-2c1c-48cc-acaa-501f4ddd558d admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:56.913708 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b1a6080f-2c1c-48cc-acaa-501f4ddd558d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54d57ba7dd0d441b9748fc600bc34972', 'request_id': 'req-b1a6080f-2c1c-48cc-acaa-501f4ddd558d'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:56.920541 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b1a6080f-2c1c-48cc-acaa-501f4ddd558d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54d57ba7dd0d441b9748fc600bc34972', 'request_id': 'req-b1a6080f-2c1c-48cc-acaa-501f4ddd558d'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:56.927625 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b1a6080f-2c1c-48cc-acaa-501f4ddd558d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54d57ba7dd0d441b9748fc600bc34972', 'request_id': 'req-b1a6080f-2c1c-48cc-acaa-501f4ddd558d'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:56.941884 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b1a6080f-2c1c-48cc-acaa-501f4ddd558d admin admin] The token cache is being invalidate because project 54d57ba7dd0d441b9748fc600bc34972 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:54:56.942064 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b1a6080f-2c1c-48cc-acaa-501f4ddd558d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:56.942797 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b1a6080f-2c1c-48cc-acaa-501f4ddd558d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:56.949690 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1362/2726] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:54:56 2022] DELETE /identity/v3/projects/54d57ba7dd0d441b9748fc600bc34972 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:54:56.955173 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fe6e3-5f53-471a-aafe-9f6dba3bea0d None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:56.955623 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fe6e3-5f53-471a-aafe-9f6dba3bea0d None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:56.955720 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fe6e3-5f53-471a-aafe-9f6dba3bea0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:57.021287 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-c29fe6e3-5f53-471a-aafe-9f6dba3bea0d None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:54:57.022412 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c29fe6e3-5f53-471a-aafe-9f6dba3bea0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:57.076253 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:54:57.091662 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1365/2727] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:54:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:54:57.100508 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22858c24-814a-4cd9-9f32-246debd6c494 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:57.101692 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22858c24-814a-4cd9-9f32-246debd6c494 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:57.101843 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22858c24-814a-4cd9-9f32-246debd6c494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:57.114379 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29fe6e3-5f53-471a-aafe-9f6dba3bea0d None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:57.116173 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c29fe6e3-5f53-471a-aafe-9f6dba3bea0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:57.128236 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-22858c24-814a-4cd9-9f32-246debd6c494 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:54:57.129445 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-22858c24-814a-4cd9-9f32-246debd6c494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:57.138184 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1363/2728] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:54:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:54:57.142387 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22858c24-814a-4cd9-9f32-246debd6c494 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:57.152713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22858c24-814a-4cd9-9f32-246debd6c494 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:57.166643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c44cf5-09fe-489d-b349-c4959a21fcae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:57.166643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c44cf5-09fe-489d-b349-c4959a21fcae admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:57.166643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c44cf5-09fe-489d-b349-c4959a21fcae admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:57.166643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c44cf5-09fe-489d-b349-c4959a21fcae admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:57.167093 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c44cf5-09fe-489d-b349-c4959a21fcae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:54:57.167093 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c44cf5-09fe-489d-b349-c4959a21fcae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:57.167093 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c44cf5-09fe-489d-b349-c4959a21fcae admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:57.174542 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1364/2729] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 14:54:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:54:57.178840 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c44cf5-09fe-489d-b349-c4959a21fcae admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:57.188976 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c44cf5-09fe-489d-b349-c4959a21fcae admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:57.193638 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcedb5-0574-49db-8c78-9763fc54c777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:57.194268 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcedb5-0574-49db-8c78-9763fc54c777 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:57.194555 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcedb5-0574-49db-8c78-9763fc54c777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:57.194836 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcedb5-0574-49db-8c78-9763fc54c777 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:57.195551 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fcedb5-0574-49db-8c78-9763fc54c777 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:57.196083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29fe6e3-5f53-471a-aafe-9f6dba3bea0d None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:57.197471 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fcedb5-0574-49db-8c78-9763fc54c777 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:57.201195 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0975a85-36aa-43a0-bf0a-ba65029309b4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4128111a9aa450abb2146a5d9926dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:57.201844 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0975a85-36aa-43a0-bf0a-ba65029309b4 service placement] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:57.202130 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0975a85-36aa-43a0-bf0a-ba65029309b4 service placement] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:57.202406 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0975a85-36aa-43a0-bf0a-ba65029309b4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:57.204214 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e0975a85-36aa-43a0-bf0a-ba65029309b4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:57.217012 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0975a85-36aa-43a0-bf0a-ba65029309b4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:57.218592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0975a85-36aa-43a0-bf0a-ba65029309b4 service placement] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:57.223406 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1365/2730] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 14:54:57 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:54:57.227378 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcedb5-0574-49db-8c78-9763fc54c777 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:57.229416 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1366/2731] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:54:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:54:57.253457 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:54:57.294399 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcedb5-0574-49db-8c78-9763fc54c777 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:57.300913 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf35a99-664c-4330-825b-5efd61f39f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:57.301685 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf35a99-664c-4330-825b-5efd61f39f6d admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:57.302097 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf35a99-664c-4330-825b-5efd61f39f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:57.302381 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf35a99-664c-4330-825b-5efd61f39f6d admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:57.303114 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf35a99-664c-4330-825b-5efd61f39f6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:57.305067 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf35a99-664c-4330-825b-5efd61f39f6d admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:57.329893 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1366/2732] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 14:54:57 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:54:57.331477 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0975a85-36aa-43a0-bf0a-ba65029309b4 service placement] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:57.344647 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0975a85-36aa-43a0-bf0a-ba65029309b4 service placement] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:57.349870 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3d6ad4-0c09-4fa0-ab3b-2aba17b8f875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:57.350618 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d6ad4-0c09-4fa0-ab3b-2aba17b8f875 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:57.350908 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d6ad4-0c09-4fa0-ab3b-2aba17b8f875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:57.351171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d6ad4-0c09-4fa0-ab3b-2aba17b8f875 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:57.351998 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3d6ad4-0c09-4fa0-ab3b-2aba17b8f875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:57.354243 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3d6ad4-0c09-4fa0-ab3b-2aba17b8f875 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:57.374181 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1367/2733] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:54:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:54:57.375684 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf35a99-664c-4330-825b-5efd61f39f6d admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:57.391864 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf35a99-664c-4330-825b-5efd61f39f6d admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:57.398782 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf925537-9a1e-42c8-86cf-87d7503683fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:57.401002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf925537-9a1e-42c8-86cf-87d7503683fa admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:57.401387 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf925537-9a1e-42c8-86cf-87d7503683fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:57.401736 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf925537-9a1e-42c8-86cf-87d7503683fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:57.402921 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf925537-9a1e-42c8-86cf-87d7503683fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:57.406605 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf925537-9a1e-42c8-86cf-87d7503683fa admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:57.419375 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1367/2734] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 14:54:57 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:54:57.423302 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3d6ad4-0c09-4fa0-ab3b-2aba17b8f875 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:57.434472 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3d6ad4-0c09-4fa0-ab3b-2aba17b8f875 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:57.439973 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3999a4-4cb5-4569-8cc2-b7621b6a69c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:57.440965 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3999a4-4cb5-4569-8cc2-b7621b6a69c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:57.441368 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3999a4-4cb5-4569-8cc2-b7621b6a69c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:57.441746 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3999a4-4cb5-4569-8cc2-b7621b6a69c1 admin admin] PATH_INFO: `/v3/projects/9ad78ee5bf484207a38d465c3499b5d4/users/1f95ff4743414cbc8cc9a213e508113b/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:57.459636 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3999a4-4cb5-4569-8cc2-b7621b6a69c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ad78ee5bf484207a38d465c3499b5d4, user_id=1f95ff4743414cbc8cc9a213e508113b, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:57.462985 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3999a4-4cb5-4569-8cc2-b7621b6a69c1 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:57.483664 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1368/2735] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 14:54:57 2022] PUT /identity/v3/projects/9ad78ee5bf484207a38d465c3499b5d4/users/1f95ff4743414cbc8cc9a213e508113b/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:54:57.489737 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d973bb-62e3-4b8f-918e-24423891992a None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:57.490035 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d973bb-62e3-4b8f-918e-24423891992a None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:57.490351 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d973bb-62e3-4b8f-918e-24423891992a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:57.511885 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-a5d973bb-62e3-4b8f-918e-24423891992a None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:54:57.513164 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-a5d973bb-62e3-4b8f-918e-24423891992a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:57.579209 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1368/2736] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:54:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:54:57.588998 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3999a4-4cb5-4569-8cc2-b7621b6a69c1 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:57.591130 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-2e3999a4-4cb5-4569-8cc2-b7621b6a69c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:57.651190 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3999a4-4cb5-4569-8cc2-b7621b6a69c1 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:57.654815 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8f9c7-c6bc-4aab-b7b6-836ed07a1916 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:57.655499 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8f9c7-c6bc-4aab-b7b6-836ed07a1916 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:57.655782 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8f9c7-c6bc-4aab-b7b6-836ed07a1916 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:57.656061 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8f9c7-c6bc-4aab-b7b6-836ed07a1916 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:57.665495 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe8f9c7-c6bc-4aab-b7b6-836ed07a1916 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:57.667084 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe8f9c7-c6bc-4aab-b7b6-836ed07a1916 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:57.677162 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1369/2737] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:54:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:55:03.012951 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6377839-20a6-4101-8922-7b97d8dd61cf None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:03.013432 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6377839-20a6-4101-8922-7b97d8dd61cf None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:03.013685 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6377839-20a6-4101-8922-7b97d8dd61cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:03.046905 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-e6377839-20a6-4101-8922-7b97d8dd61cf None None] MFA Rules not processed for user `1f95ff4743414cbc8cc9a213e508113b`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:03.048466 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e6377839-20a6-4101-8922-7b97d8dd61cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:03.120940 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1369/2738] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:55:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:03.135947 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8f9c7-c6bc-4aab-b7b6-836ed07a1916 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:03.137549 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1fe8f9c7-c6bc-4aab-b7b6-836ed07a1916 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:03.194355 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:03.219119 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8f9c7-c6bc-4aab-b7b6-836ed07a1916 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:03.224577 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828adfd0-ed24-455d-b3a8-34c7e3c7244e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:03.225234 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828adfd0-ed24-455d-b3a8-34c7e3c7244e service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:03.225518 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828adfd0-ed24-455d-b3a8-34c7e3c7244e service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:03.225798 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828adfd0-ed24-455d-b3a8-34c7e3c7244e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:03.235751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828adfd0-ed24-455d-b3a8-34c7e3c7244e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:03.237359 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828adfd0-ed24-455d-b3a8-34c7e3c7244e service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:03.247397 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1370/2739] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:55:04.530042 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23abf94-e0d3-4ba4-9e02-01a461b8deb0 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:04.530608 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23abf94-e0d3-4ba4-9e02-01a461b8deb0 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:04.530887 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23abf94-e0d3-4ba4-9e02-01a461b8deb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:04.552417 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-e23abf94-e0d3-4ba4-9e02-01a461b8deb0 None None] MFA Rules not processed for user `1f95ff4743414cbc8cc9a213e508113b`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:04.553862 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e23abf94-e0d3-4ba4-9e02-01a461b8deb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:04.561943 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1370/2740] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:55:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:08.264940 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828adfd0-ed24-455d-b3a8-34c7e3c7244e service nova] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:08.267024 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-828adfd0-ed24-455d-b3a8-34c7e3c7244e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:08.292577 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828adfd0-ed24-455d-b3a8-34c7e3c7244e service nova] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:08.298777 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90f298-ae5f-4062-85d5-4314d34741ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:08.299497 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90f298-ae5f-4062-85d5-4314d34741ec service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:08.299781 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90f298-ae5f-4062-85d5-4314d34741ec service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:08.300063 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90f298-ae5f-4062-85d5-4314d34741ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:08.301961 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-4c90f298-ae5f-4062-85d5-4314d34741ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:08.322550 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90f298-ae5f-4062-85d5-4314d34741ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:08.324229 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90f298-ae5f-4062-85d5-4314d34741ec service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:08.337235 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1371/2741] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:55:14.545874 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=86,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:18.330987 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=20 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:19.829111 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23abf94-e0d3-4ba4-9e02-01a461b8deb0 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:19.846154 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23abf94-e0d3-4ba4-9e02-01a461b8deb0 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:19.849807 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ac078-6ab1-476f-833b-36cee0009d10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:19.850460 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ac078-6ab1-476f-833b-36cee0009d10 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:19.850745 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ac078-6ab1-476f-833b-36cee0009d10 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:19.851050 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ac078-6ab1-476f-833b-36cee0009d10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:19.852882 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b25ac078-6ab1-476f-833b-36cee0009d10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:19.865714 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ac078-6ab1-476f-833b-36cee0009d10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:19.867383 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ac078-6ab1-476f-833b-36cee0009d10 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:19.878762 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1371/2742] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:55:25.406787 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90f298-ae5f-4062-85d5-4314d34741ec service nova] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:25.416820 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90f298-ae5f-4062-85d5-4314d34741ec service nova] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:25.420467 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dbb550-dd9c-469c-a372-d8de48c4608b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:25.421107 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbb550-dd9c-469c-a372-d8de48c4608b service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:25.421390 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbb550-dd9c-469c-a372-d8de48c4608b service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:25.421666 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbb550-dd9c-469c-a372-d8de48c4608b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:25.423472 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a8dbb550-dd9c-469c-a372-d8de48c4608b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:25.484878 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dbb550-dd9c-469c-a372-d8de48c4608b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:25.486310 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dbb550-dd9c-469c-a372-d8de48c4608b service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:25.495821 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1372/2743] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:25 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:55:26.333851 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ac078-6ab1-476f-833b-36cee0009d10 service nova] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:26.335470 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b25ac078-6ab1-476f-833b-36cee0009d10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:26.354227 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ac078-6ab1-476f-833b-36cee0009d10 service nova] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:26.358946 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a99fc-28de-4ed0-8afd-84cec1cf2914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:26.359850 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a99fc-28de-4ed0-8afd-84cec1cf2914 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:26.360138 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a99fc-28de-4ed0-8afd-84cec1cf2914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:26.360417 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a99fc-28de-4ed0-8afd-84cec1cf2914 admin admin] PATH_INFO: `/v3/users/235cd365a5ab41cda9bffd0cee150617` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:26.361639 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896a99fc-28de-4ed0-8afd-84cec1cf2914 admin admin] RBAC: Authorizing `identity:delete_user(user_id=235cd365a5ab41cda9bffd0cee150617)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:26.363581 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896a99fc-28de-4ed0-8afd-84cec1cf2914 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:26.410897 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-896a99fc-28de-4ed0-8afd-84cec1cf2914 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '235cd365a5ab41cda9bffd0cee150617'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:26.411587 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-896a99fc-28de-4ed0-8afd-84cec1cf2914 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '235cd365a5ab41cda9bffd0cee150617'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:26.424523 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-896a99fc-28de-4ed0-8afd-84cec1cf2914 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '235cd365a5ab41cda9bffd0cee150617'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:26.432922 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-896a99fc-28de-4ed0-8afd-84cec1cf2914 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '235cd365a5ab41cda9bffd0cee150617'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:26.440099 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-896a99fc-28de-4ed0-8afd-84cec1cf2914 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '235cd365a5ab41cda9bffd0cee150617'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:26.446404 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1372/2744] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 14:55:26 2022] DELETE /identity/v3/users/235cd365a5ab41cda9bffd0cee150617 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:55:26.621962 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dbb550-dd9c-469c-a372-d8de48c4608b service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:26.633814 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dbb550-dd9c-469c-a372-d8de48c4608b service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:26.697133 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc042444-c6ca-4714-9118-4751e796a135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:26.698028 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc042444-c6ca-4714-9118-4751e796a135 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:26.698309 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc042444-c6ca-4714-9118-4751e796a135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:26.698584 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc042444-c6ca-4714-9118-4751e796a135 admin admin] PATH_INFO: `/v3/projects/28dfd45278d84a798da8e77bf7e6bc9d` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:26.699814 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc042444-c6ca-4714-9118-4751e796a135 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28dfd45278d84a798da8e77bf7e6bc9d)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:26.701727 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc042444-c6ca-4714-9118-4751e796a135 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:26.754690 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-cc042444-c6ca-4714-9118-4751e796a135 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28dfd45278d84a798da8e77bf7e6bc9d', 'request_id': 'req-cc042444-c6ca-4714-9118-4751e796a135'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:26.763318 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-cc042444-c6ca-4714-9118-4751e796a135 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28dfd45278d84a798da8e77bf7e6bc9d', 'request_id': 'req-cc042444-c6ca-4714-9118-4751e796a135'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:26.771040 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-cc042444-c6ca-4714-9118-4751e796a135 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28dfd45278d84a798da8e77bf7e6bc9d', 'request_id': 'req-cc042444-c6ca-4714-9118-4751e796a135'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:26.783734 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-cc042444-c6ca-4714-9118-4751e796a135 admin admin] The token cache is being invalidate because project 28dfd45278d84a798da8e77bf7e6bc9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:55:26.784022 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-cc042444-c6ca-4714-9118-4751e796a135 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:26.784672 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-cc042444-c6ca-4714-9118-4751e796a135 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:26.786231 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1373/2745] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 14:55:26 2022] DELETE /identity/v3/projects/28dfd45278d84a798da8e77bf7e6bc9d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:55:36.442484 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=35,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:36.780793 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=81,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:52.601904 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19245843-e609-4dec-b29f-90daf2ce4edf None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:52.602365 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19245843-e609-4dec-b29f-90daf2ce4edf None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:52.602472 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19245843-e609-4dec-b29f-90daf2ce4edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:52.620932 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-19245843-e609-4dec-b29f-90daf2ce4edf None None] MFA Rules not processed for user `1f95ff4743414cbc8cc9a213e508113b`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:52.622252 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-19245843-e609-4dec-b29f-90daf2ce4edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:52.676885 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1373/2746] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:56:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:52.978007 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc042444-c6ca-4714-9118-4751e796a135 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:52.979480 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-cc042444-c6ca-4714-9118-4751e796a135 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:53.035043 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc042444-c6ca-4714-9118-4751e796a135 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:53.038564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ffaa7-d4e1-4e37-b95b-e1493b4c8209 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:53.039240 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ffaa7-d4e1-4e37-b95b-e1493b4c8209 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:53.039524 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ffaa7-d4e1-4e37-b95b-e1493b4c8209 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:53.039798 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ffaa7-d4e1-4e37-b95b-e1493b4c8209 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:53.042824 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-398ffaa7-d4e1-4e37-b95b-e1493b4c8209 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:53.052619 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398ffaa7-d4e1-4e37-b95b-e1493b4c8209 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:53.054041 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398ffaa7-d4e1-4e37-b95b-e1493b4c8209 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:53.062630 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1374/2747] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:53.874975 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19245843-e609-4dec-b29f-90daf2ce4edf None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:53.876451 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-19245843-e609-4dec-b29f-90daf2ce4edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:53.931308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ffaa7-d4e1-4e37-b95b-e1493b4c8209 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:53.932897 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-398ffaa7-d4e1-4e37-b95b-e1493b4c8209 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:53.969869 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19245843-e609-4dec-b29f-90daf2ce4edf None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:53.974913 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b298c7-9f65-4dcb-882e-6d6403994c9b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4128111a9aa450abb2146a5d9926dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:53.975586 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b298c7-9f65-4dcb-882e-6d6403994c9b service placement] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:53.975878 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b298c7-9f65-4dcb-882e-6d6403994c9b service placement] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:53.976161 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b298c7-9f65-4dcb-882e-6d6403994c9b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:53.977909 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-94b298c7-9f65-4dcb-882e-6d6403994c9b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:53.989482 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b298c7-9f65-4dcb-882e-6d6403994c9b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:53.991075 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b298c7-9f65-4dcb-882e-6d6403994c9b service placement] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:54.001405 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1374/2748] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:56:54.003040 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ffaa7-d4e1-4e37-b95b-e1493b4c8209 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:54.008148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2accef4d-addb-45b3-a276-b01370a31ab5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4128111a9aa450abb2146a5d9926dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:54.008799 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2accef4d-addb-45b3-a276-b01370a31ab5 service placement] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:54.009202 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2accef4d-addb-45b3-a276-b01370a31ab5 service placement] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:54.009485 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2accef4d-addb-45b3-a276-b01370a31ab5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:54.019259 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2accef4d-addb-45b3-a276-b01370a31ab5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:54.020984 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2accef4d-addb-45b3-a276-b01370a31ab5 service placement] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:54.031242 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1375/2749] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:57:03.996658 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=66 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:04.026814 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=58 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:58.950249 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f8b7b-f202-4772-aae9-ec8f3cff71dc None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:58.950846 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f8b7b-f202-4772-aae9-ec8f3cff71dc None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:58.950846 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f8b7b-f202-4772-aae9-ec8f3cff71dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:58.969183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-c97f8b7b-f202-4772-aae9-ec8f3cff71dc None None] MFA Rules not processed for user `1b23fa0b7fdb4016917987ea1c40af33`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:58.970356 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c97f8b7b-f202-4772-aae9-ec8f3cff71dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:59.023582 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1375/2750] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:57:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:58:09.017092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=25 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:58:22.804877 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2accef4d-addb-45b3-a276-b01370a31ab5 service placement] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:22.814246 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2accef4d-addb-45b3-a276-b01370a31ab5 service placement] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:22.817808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b423478-480d-4658-b2cf-ed628e8268ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:22.818453 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b423478-480d-4658-b2cf-ed628e8268ae service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:22.818731 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b423478-480d-4658-b2cf-ed628e8268ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:22.819152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b423478-480d-4658-b2cf-ed628e8268ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:22.820886 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5b423478-480d-4658-b2cf-ed628e8268ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:22.901342 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b423478-480d-4658-b2cf-ed628e8268ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:22.902828 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b423478-480d-4658-b2cf-ed628e8268ae service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:22.923266 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1376/2751] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:58:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:58:32.920642 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=43 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:58:34.915392 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e4e869-dbd4-4cfd-86dc-a549026936c5 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:34.916219 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e4e869-dbd4-4cfd-86dc-a549026936c5 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:34.916219 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e4e869-dbd4-4cfd-86dc-a549026936c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:34.934360 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-59e4e869-dbd4-4cfd-86dc-a549026936c5 None None] MFA Rules not processed for user `1f95ff4743414cbc8cc9a213e508113b`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:58:34.935783 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-59e4e869-dbd4-4cfd-86dc-a549026936c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:34.943148 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1376/2752] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 14:58:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:58:44.928625 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:56.581382 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b423478-480d-4658-b2cf-ed628e8268ae service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:56.583046 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5b423478-480d-4658-b2cf-ed628e8268ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:56.593786 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b423478-480d-4658-b2cf-ed628e8268ae service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:56.597380 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe37e2-9d4e-4920-9e0d-afcc4562d58b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:56.598036 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe37e2-9d4e-4920-9e0d-afcc4562d58b service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:56.598345 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe37e2-9d4e-4920-9e0d-afcc4562d58b service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:56.598619 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe37e2-9d4e-4920-9e0d-afcc4562d58b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:56.600338 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-7efe37e2-9d4e-4920-9e0d-afcc4562d58b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:56.609502 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efe37e2-9d4e-4920-9e0d-afcc4562d58b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:56.610942 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efe37e2-9d4e-4920-9e0d-afcc4562d58b service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:56.619429 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1377/2753] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 14:59:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:00:06.615851 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=6 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:00:08.461515 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e4e869-dbd4-4cfd-86dc-a549026936c5 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:00:08.462890 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-59e4e869-dbd4-4cfd-86dc-a549026936c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:08.544901 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e4e869-dbd4-4cfd-86dc-a549026936c5 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:00:08.548670 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b3fb01-d509-4d73-baae-971e9e2253b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:00:08.549316 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3fb01-d509-4d73-baae-971e9e2253b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:00:08.549601 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3fb01-d509-4d73-baae-971e9e2253b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:00:08.551022 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3fb01-d509-4d73-baae-971e9e2253b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:00:08.553242 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-93b3fb01-d509-4d73-baae-971e9e2253b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:08.564202 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b3fb01-d509-4d73-baae-971e9e2253b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:00:08.565628 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b3fb01-d509-4d73-baae-971e9e2253b2 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:00:08.576139 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1377/2754] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 15:00:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:00:15.267013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe37e2-9d4e-4920-9e0d-afcc4562d58b service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:00:15.268230 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-7efe37e2-9d4e-4920-9e0d-afcc4562d58b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:15.284592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe37e2-9d4e-4920-9e0d-afcc4562d58b service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:00:15.290184 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5612f3-5938-4112-ba82-e3f85fac34c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:00:15.290834 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5612f3-5938-4112-ba82-e3f85fac34c6 service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:00:15.291154 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5612f3-5938-4112-ba82-e3f85fac34c6 service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:00:15.291437 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5612f3-5938-4112-ba82-e3f85fac34c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:00:15.293476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-fe5612f3-5938-4112-ba82-e3f85fac34c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:15.304743 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5612f3-5938-4112-ba82-e3f85fac34c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:00:15.306326 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5612f3-5938-4112-ba82-e3f85fac34c6 service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:00:15.315260 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1378/2755] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 15:00:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:00:18.570439 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=35 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:00:20.599547 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54145354-6763-4ce7-a018-61e4dba01e2a None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:00:20.600185 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54145354-6763-4ce7-a018-61e4dba01e2a None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:00:20.600185 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54145354-6763-4ce7-a018-61e4dba01e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:00:20.618891 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-54145354-6763-4ce7-a018-61e4dba01e2a None None] MFA Rules not processed for user `1b23fa0b7fdb4016917987ea1c40af33`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:00:20.620135 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-54145354-6763-4ce7-a018-61e4dba01e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:20.626912 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1378/2756] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 15:00:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:00:20.666194 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5612f3-5938-4112-ba82-e3f85fac34c6 service nova] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:00:20.667777 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-fe5612f3-5938-4112-ba82-e3f85fac34c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:20.679957 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5612f3-5938-4112-ba82-e3f85fac34c6 service nova] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:00:20.683470 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be5135c-74e4-4dd5-b9fa-24ed1439a9cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:00:20.684102 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5135c-74e4-4dd5-b9fa-24ed1439a9cf service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:00:20.684384 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5135c-74e4-4dd5-b9fa-24ed1439a9cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:00:20.684665 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5135c-74e4-4dd5-b9fa-24ed1439a9cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:00:20.686352 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-3be5135c-74e4-4dd5-b9fa-24ed1439a9cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:20.700152 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be5135c-74e4-4dd5-b9fa-24ed1439a9cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:00:20.701604 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be5135c-74e4-4dd5-b9fa-24ed1439a9cf service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:00:20.712718 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1379/2757] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 15:00:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:00:21.006948 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54145354-6763-4ce7-a018-61e4dba01e2a None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:00:21.017106 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54145354-6763-4ce7-a018-61e4dba01e2a None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:00:21.021431 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e979f-05ae-4d02-b76f-5aebc2a8aa9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:00:21.022088 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e979f-05ae-4d02-b76f-5aebc2a8aa9b service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:00:21.022369 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e979f-05ae-4d02-b76f-5aebc2a8aa9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:00:21.022643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e979f-05ae-4d02-b76f-5aebc2a8aa9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:00:21.024808 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-2e5e979f-05ae-4d02-b76f-5aebc2a8aa9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:21.034960 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5e979f-05ae-4d02-b76f-5aebc2a8aa9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:00:21.037150 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5e979f-05ae-4d02-b76f-5aebc2a8aa9b service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:00:21.045694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1379/2758] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 15:00:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:00:21.669661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be5135c-74e4-4dd5-b9fa-24ed1439a9cf service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:00:21.681112 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be5135c-74e4-4dd5-b9fa-24ed1439a9cf service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:00:21.686095 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be5a689-bfc4-408a-b0f6-d54cc113002d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4128111a9aa450abb2146a5d9926dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:00:21.686749 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be5a689-bfc4-408a-b0f6-d54cc113002d service placement] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:00:21.687063 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be5a689-bfc4-408a-b0f6-d54cc113002d service placement] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:00:21.687405 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be5a689-bfc4-408a-b0f6-d54cc113002d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:00:21.689314 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1be5a689-bfc4-408a-b0f6-d54cc113002d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:21.702083 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be5a689-bfc4-408a-b0f6-d54cc113002d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:00:21.703735 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be5a689-bfc4-408a-b0f6-d54cc113002d service placement] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:00:21.716141 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1380/2759] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 15:00:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:00:24.033056 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e979f-05ae-4d02-b76f-5aebc2a8aa9b service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:00:24.034569 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-2e5e979f-05ae-4d02-b76f-5aebc2a8aa9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:24.052417 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e979f-05ae-4d02-b76f-5aebc2a8aa9b service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:00:24.059557 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14b995-8630-4d57-8b4d-be84cbf3592a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:00:24.060367 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14b995-8630-4d57-8b4d-be84cbf3592a service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:00:24.060812 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14b995-8630-4d57-8b4d-be84cbf3592a service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:00:24.061324 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14b995-8630-4d57-8b4d-be84cbf3592a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:00:24.063795 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-0a14b995-8630-4d57-8b4d-be84cbf3592a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:24.079043 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a14b995-8630-4d57-8b4d-be84cbf3592a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:00:24.081035 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a14b995-8630-4d57-8b4d-be84cbf3592a service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:00:24.093979 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1380/2760] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 15:00:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:00:31.711003 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=18 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:00:34.088575 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=15 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:00:39.274830 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be5a689-bfc4-408a-b0f6-d54cc113002d service placement] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:00:39.292515 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be5a689-bfc4-408a-b0f6-d54cc113002d service placement] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:00:39.299425 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef967a4-8d56-484a-b259-7ad628494df7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:00:39.300192 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef967a4-8d56-484a-b259-7ad628494df7 service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:00:39.300478 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef967a4-8d56-484a-b259-7ad628494df7 service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:00:39.300976 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef967a4-8d56-484a-b259-7ad628494df7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:00:39.302854 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-7ef967a4-8d56-484a-b259-7ad628494df7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:39.317220 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef967a4-8d56-484a-b259-7ad628494df7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:00:39.319539 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef967a4-8d56-484a-b259-7ad628494df7 service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:00:39.336726 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef6672-95ac-4920-a927-d99cd066dd7d None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:00:39.337253 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef6672-95ac-4920-a927-d99cd066dd7d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:00:39.337380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef6672-95ac-4920-a927-d99cd066dd7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:00:39.340243 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1381/2761] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 15:00:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:00:39.359426 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-59ef6672-95ac-4920-a927-d99cd066dd7d None None] MFA Rules not processed for user `1f95ff4743414cbc8cc9a213e508113b`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:00:39.360811 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-59ef6672-95ac-4920-a927-d99cd066dd7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:39.371019 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1381/2762] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 15:00:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:00:41.904023 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef967a4-8d56-484a-b259-7ad628494df7 service nova] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:00:41.914458 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef967a4-8d56-484a-b259-7ad628494df7 service nova] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:00:41.918754 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666ef829-cd6a-450d-9f0d-7e4a2c27aa8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:00:41.919546 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ef829-cd6a-450d-9f0d-7e4a2c27aa8e service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:00:41.919825 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ef829-cd6a-450d-9f0d-7e4a2c27aa8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:00:41.920629 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ef829-cd6a-450d-9f0d-7e4a2c27aa8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:00:41.923207 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-666ef829-cd6a-450d-9f0d-7e4a2c27aa8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:42.000043 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666ef829-cd6a-450d-9f0d-7e4a2c27aa8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:00:42.001496 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666ef829-cd6a-450d-9f0d-7e4a2c27aa8e service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:00:42.011943 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1382/2763] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 15:00:41 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:00:49.353884 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:00:52.007410 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=36 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:00:53.975928 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ef6672-95ac-4920-a927-d99cd066dd7d None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:00:53.991853 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ef6672-95ac-4920-a927-d99cd066dd7d None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:00:53.998358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be01c228-2d7f-42c3-a70e-34ce5bc219af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:00:53.999250 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01c228-2d7f-42c3-a70e-34ce5bc219af service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:00:53.999616 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01c228-2d7f-42c3-a70e-34ce5bc219af service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:00:53.999977 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01c228-2d7f-42c3-a70e-34ce5bc219af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:00:54.002187 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-be01c228-2d7f-42c3-a70e-34ce5bc219af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:54.016408 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be01c228-2d7f-42c3-a70e-34ce5bc219af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:00:54.018280 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be01c228-2d7f-42c3-a70e-34ce5bc219af service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:00:54.029186 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1382/2764] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 15:00:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:01:04.024540 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=6 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:01:58.631739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666ef829-cd6a-450d-9f0d-7e4a2c27aa8e service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:58.643448 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666ef829-cd6a-450d-9f0d-7e4a2c27aa8e service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:58.648479 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d259e509-8ce4-4110-9f00-8ad751d27170 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4128111a9aa450abb2146a5d9926dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:58.649114 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259e509-8ce4-4110-9f00-8ad751d27170 service placement] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:58.649399 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259e509-8ce4-4110-9f00-8ad751d27170 service placement] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:58.649675 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259e509-8ce4-4110-9f00-8ad751d27170 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:58.659144 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d259e509-8ce4-4110-9f00-8ad751d27170 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:58.660721 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d259e509-8ce4-4110-9f00-8ad751d27170 service placement] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:58.671585 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1383/2765] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 15:01:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:02:08.666406 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=6 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:43.489043 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be01c228-2d7f-42c3-a70e-34ce5bc219af service nova] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:43.498381 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be01c228-2d7f-42c3-a70e-34ce5bc219af service nova] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:43.502105 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b158c3c-6a7d-4d71-b5d0-71394130c803 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:43.502748 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b158c3c-6a7d-4d71-b5d0-71394130c803 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:43.503076 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b158c3c-6a7d-4d71-b5d0-71394130c803 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:43.503362 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b158c3c-6a7d-4d71-b5d0-71394130c803 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:43.505155 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-6b158c3c-6a7d-4d71-b5d0-71394130c803 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:43.566789 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b158c3c-6a7d-4d71-b5d0-71394130c803 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:43.568254 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b158c3c-6a7d-4d71-b5d0-71394130c803 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:43.577657 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1383/2766] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 15:02:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:02:46.351693 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d259e509-8ce4-4110-9f00-8ad751d27170 service placement] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:46.365443 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d259e509-8ce4-4110-9f00-8ad751d27170 service placement] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:46.372898 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d3f98-6198-4d24-b927-f3d15298fcb5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4128111a9aa450abb2146a5d9926dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:46.373798 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d3f98-6198-4d24-b927-f3d15298fcb5 service placement] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:46.374087 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d3f98-6198-4d24-b927-f3d15298fcb5 service placement] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:46.374436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d3f98-6198-4d24-b927-f3d15298fcb5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:46.376297 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-fb9d3f98-6198-4d24-b927-f3d15298fcb5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:46.397232 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d3f98-6198-4d24-b927-f3d15298fcb5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:46.399127 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d3f98-6198-4d24-b927-f3d15298fcb5 service placement] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:46.412417 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1384/2767] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 15:02:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:02:46.416988 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b158c3c-6a7d-4d71-b5d0-71394130c803 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:46.418658 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-6b158c3c-6a7d-4d71-b5d0-71394130c803 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:46.462768 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b158c3c-6a7d-4d71-b5d0-71394130c803 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:46.468519 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd250199-3a89-4e5b-85e1-ebf5a8bf9296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:46.469395 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd250199-3a89-4e5b-85e1-ebf5a8bf9296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:46.469681 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd250199-3a89-4e5b-85e1-ebf5a8bf9296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:46.469962 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd250199-3a89-4e5b-85e1-ebf5a8bf9296 admin admin] PATH_INFO: `/v3/users/1f95ff4743414cbc8cc9a213e508113b` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:46.471206 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd250199-3a89-4e5b-85e1-ebf5a8bf9296 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f95ff4743414cbc8cc9a213e508113b)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:46.473181 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd250199-3a89-4e5b-85e1-ebf5a8bf9296 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:46.519638 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-bd250199-3a89-4e5b-85e1-ebf5a8bf9296 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f95ff4743414cbc8cc9a213e508113b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:46.520311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-bd250199-3a89-4e5b-85e1-ebf5a8bf9296 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f95ff4743414cbc8cc9a213e508113b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:46.532974 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-bd250199-3a89-4e5b-85e1-ebf5a8bf9296 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f95ff4743414cbc8cc9a213e508113b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:46.539652 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-bd250199-3a89-4e5b-85e1-ebf5a8bf9296 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f95ff4743414cbc8cc9a213e508113b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:46.547042 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-bd250199-3a89-4e5b-85e1-ebf5a8bf9296 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f95ff4743414cbc8cc9a213e508113b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:46.553159 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1384/2768] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 15:02:46 2022] DELETE /identity/v3/users/1f95ff4743414cbc8cc9a213e508113b => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:46.734879 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d3f98-6198-4d24-b927-f3d15298fcb5 service placement] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:46.745719 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d3f98-6198-4d24-b927-f3d15298fcb5 service placement] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:46.802448 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5baed78-99c2-41cb-bffe-f68e7f56bd78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:46.803343 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5baed78-99c2-41cb-bffe-f68e7f56bd78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:46.803630 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5baed78-99c2-41cb-bffe-f68e7f56bd78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:46.803912 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5baed78-99c2-41cb-bffe-f68e7f56bd78 admin admin] PATH_INFO: `/v3/projects/9ad78ee5bf484207a38d465c3499b5d4` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:46.805089 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5baed78-99c2-41cb-bffe-f68e7f56bd78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ad78ee5bf484207a38d465c3499b5d4)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:46.807030 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5baed78-99c2-41cb-bffe-f68e7f56bd78 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:46.852549 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d5baed78-99c2-41cb-bffe-f68e7f56bd78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ad78ee5bf484207a38d465c3499b5d4', 'request_id': 'req-d5baed78-99c2-41cb-bffe-f68e7f56bd78'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:46.859512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d5baed78-99c2-41cb-bffe-f68e7f56bd78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ad78ee5bf484207a38d465c3499b5d4', 'request_id': 'req-d5baed78-99c2-41cb-bffe-f68e7f56bd78'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:46.865755 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d5baed78-99c2-41cb-bffe-f68e7f56bd78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ad78ee5bf484207a38d465c3499b5d4', 'request_id': 'req-d5baed78-99c2-41cb-bffe-f68e7f56bd78'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:46.875993 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d5baed78-99c2-41cb-bffe-f68e7f56bd78 admin admin] The token cache is being invalidate because project 9ad78ee5bf484207a38d465c3499b5d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:02:46.876283 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d5baed78-99c2-41cb-bffe-f68e7f56bd78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:46.876919 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d5baed78-99c2-41cb-bffe-f68e7f56bd78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:46.878537 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1385/2769] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 15:02:46 2022] DELETE /identity/v3/projects/9ad78ee5bf484207a38d465c3499b5d4 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:46.883460 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead3bd0-24ae-4594-ab90-572888227a7b None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:46.883753 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead3bd0-24ae-4594-ab90-572888227a7b None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:46.884031 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead3bd0-24ae-4594-ab90-572888227a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:46.901856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-dead3bd0-24ae-4594-ab90-572888227a7b None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:46.903707 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-dead3bd0-24ae-4594-ab90-572888227a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:46.962670 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1385/2770] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 15:02:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:46.971735 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72849176-31d6-465c-b28f-b49a319470b8 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:46.972049 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72849176-31d6-465c-b28f-b49a319470b8 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:46.972329 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72849176-31d6-465c-b28f-b49a319470b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:46.990121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-72849176-31d6-465c-b28f-b49a319470b8 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:46.991363 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-72849176-31d6-465c-b28f-b49a319470b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:46.999275 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1386/2771] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 15:02:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:47.003283 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead3bd0-24ae-4594-ab90-572888227a7b None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:47.013334 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead3bd0-24ae-4594-ab90-572888227a7b None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:47.017849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf51a54-da08-44c0-b860-2918f8016694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:47.018497 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf51a54-da08-44c0-b860-2918f8016694 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:47.018784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf51a54-da08-44c0-b860-2918f8016694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:47.019101 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf51a54-da08-44c0-b860-2918f8016694 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:47.019825 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf51a54-da08-44c0-b860-2918f8016694 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:02:47.020135 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf51a54-da08-44c0-b860-2918f8016694 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:47.021728 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf51a54-da08-44c0-b860-2918f8016694 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:47.023324 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:47.036031 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1386/2772] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 15:02:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:02:47.040096 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72849176-31d6-465c-b28f-b49a319470b8 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:47.049600 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72849176-31d6-465c-b28f-b49a319470b8 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:47.054238 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408898cd-29e8-46ae-b214-659568a60910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:47.054881 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408898cd-29e8-46ae-b214-659568a60910 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:47.055202 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408898cd-29e8-46ae-b214-659568a60910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:47.055483 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408898cd-29e8-46ae-b214-659568a60910 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:47.056174 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408898cd-29e8-46ae-b214-659568a60910 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:47.058075 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408898cd-29e8-46ae-b214-659568a60910 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:47.092876 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1387/2773] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 15:02:47 2022] POST /identity/v3/projects => generated 403 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:47.096894 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf51a54-da08-44c0-b860-2918f8016694 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:47.106687 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf51a54-da08-44c0-b860-2918f8016694 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:47.162315 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756242f1-00a1-463a-9908-4247b90b2a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:47.162983 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756242f1-00a1-463a-9908-4247b90b2a98 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:47.163301 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756242f1-00a1-463a-9908-4247b90b2a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:47.163580 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756242f1-00a1-463a-9908-4247b90b2a98 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:47.164264 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756242f1-00a1-463a-9908-4247b90b2a98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:47.166169 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756242f1-00a1-463a-9908-4247b90b2a98 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:47.186208 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1387/2774] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 15:02:47 2022] POST /identity/v3/users => generated 302 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:47.190272 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408898cd-29e8-46ae-b214-659568a60910 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:47.200955 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408898cd-29e8-46ae-b214-659568a60910 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:47.205624 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca76a3-ede2-485d-aece-c4a26b33bb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:47.206267 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca76a3-ede2-485d-aece-c4a26b33bb19 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:47.206549 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca76a3-ede2-485d-aece-c4a26b33bb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:47.206824 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca76a3-ede2-485d-aece-c4a26b33bb19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:47.207593 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ca76a3-ede2-485d-aece-c4a26b33bb19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:47.209178 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ca76a3-ede2-485d-aece-c4a26b33bb19 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:47.218913 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1388/2775] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:47.222562 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756242f1-00a1-463a-9908-4247b90b2a98 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:47.232503 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756242f1-00a1-463a-9908-4247b90b2a98 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:47.237172 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363113e2-4825-4e94-a0cd-9b97bf3e6262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:47.237849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363113e2-4825-4e94-a0cd-9b97bf3e6262 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:47.238135 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363113e2-4825-4e94-a0cd-9b97bf3e6262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:47.238414 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363113e2-4825-4e94-a0cd-9b97bf3e6262 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:47.239169 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363113e2-4825-4e94-a0cd-9b97bf3e6262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:47.240910 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363113e2-4825-4e94-a0cd-9b97bf3e6262 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:47.251464 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1388/2776] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:47.255474 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca76a3-ede2-485d-aece-c4a26b33bb19 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:47.265813 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca76a3-ede2-485d-aece-c4a26b33bb19 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:47.270467 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c61248-58a5-431f-ace5-53deb28f7f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:47.271369 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c61248-58a5-431f-ace5-53deb28f7f34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:47.271651 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c61248-58a5-431f-ace5-53deb28f7f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:47.271927 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c61248-58a5-431f-ace5-53deb28f7f34 admin admin] PATH_INFO: `/v3/projects/63bcfcfec16e409294be13f25820f8c6/users/5182756ca21049d0b570337552292310/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:47.286267 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c61248-58a5-431f-ace5-53deb28f7f34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63bcfcfec16e409294be13f25820f8c6, user_id=5182756ca21049d0b570337552292310, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:47.288935 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c61248-58a5-431f-ace5-53deb28f7f34 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:47.302524 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1389/2777] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 15:02:47 2022] PUT /identity/v3/projects/63bcfcfec16e409294be13f25820f8c6/users/5182756ca21049d0b570337552292310/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:47.313012 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912dabac-3b1c-4b86-b471-7755b1f26de6 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:47.313309 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912dabac-3b1c-4b86-b471-7755b1f26de6 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:47.313594 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912dabac-3b1c-4b86-b471-7755b1f26de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:47.340167 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-912dabac-3b1c-4b86-b471-7755b1f26de6 None None] MFA Rules not processed for user `5182756ca21049d0b570337552292310`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:47.341380 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-912dabac-3b1c-4b86-b471-7755b1f26de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:47.403839 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1389/2778] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 15:02:47 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:47.408264 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c61248-58a5-431f-ace5-53deb28f7f34 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:47.413788 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:47.471238 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c61248-58a5-431f-ace5-53deb28f7f34 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:47.475900 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c672f-300a-41f1-8e79-afb2569b6320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:47.476557 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c672f-300a-41f1-8e79-afb2569b6320 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:47.476842 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c672f-300a-41f1-8e79-afb2569b6320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:47.477116 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c672f-300a-41f1-8e79-afb2569b6320 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:47.477797 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c672f-300a-41f1-8e79-afb2569b6320 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:47.479883 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c672f-300a-41f1-8e79-afb2569b6320 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:47.505723 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1390/2779] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 15:02:47 2022] POST /identity/v3/projects => generated 405 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:47.510018 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912dabac-3b1c-4b86-b471-7755b1f26de6 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:47.521242 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912dabac-3b1c-4b86-b471-7755b1f26de6 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:47.581340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c0a2b-2d1b-44d1-8324-7a6eb9b6a5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:47.582014 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c0a2b-2d1b-44d1-8324-7a6eb9b6a5a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:47.582304 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c0a2b-2d1b-44d1-8324-7a6eb9b6a5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:47.582581 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c0a2b-2d1b-44d1-8324-7a6eb9b6a5a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:47.583314 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7c0a2b-2d1b-44d1-8324-7a6eb9b6a5a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:47.585257 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7c0a2b-2d1b-44d1-8324-7a6eb9b6a5a7 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:47.593302 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:47.605834 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1390/2780] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 15:02:47 2022] POST /identity/v3/users => generated 309 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:47.610019 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c672f-300a-41f1-8e79-afb2569b6320 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:47.620451 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c672f-300a-41f1-8e79-afb2569b6320 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:47.625206 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09882e36-211f-4478-aff0-65dbc793c8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:47.625866 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09882e36-211f-4478-aff0-65dbc793c8d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:47.626157 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09882e36-211f-4478-aff0-65dbc793c8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:47.626438 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09882e36-211f-4478-aff0-65dbc793c8d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:47.627208 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09882e36-211f-4478-aff0-65dbc793c8d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:47.628827 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09882e36-211f-4478-aff0-65dbc793c8d6 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:47.641229 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1391/2781] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:47.645507 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c0a2b-2d1b-44d1-8324-7a6eb9b6a5a7 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:47.656186 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c0a2b-2d1b-44d1-8324-7a6eb9b6a5a7 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:47.661088 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6374f-3974-4a40-8063-360ed6cd5c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:47.661960 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6374f-3974-4a40-8063-360ed6cd5c35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:47.662239 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6374f-3974-4a40-8063-360ed6cd5c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:47.662516 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6374f-3974-4a40-8063-360ed6cd5c35 admin admin] PATH_INFO: `/v3/projects/4c9e146ca7b849a79a8ad989a215f922/users/94b4e11f7a974ceebdda0ef51be67220/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:47.677529 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6374f-3974-4a40-8063-360ed6cd5c35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c9e146ca7b849a79a8ad989a215f922, user_id=94b4e11f7a974ceebdda0ef51be67220, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:47.680276 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6374f-3974-4a40-8063-360ed6cd5c35 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:47.693039 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1391/2782] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 15:02:47 2022] PUT /identity/v3/projects/4c9e146ca7b849a79a8ad989a215f922/users/94b4e11f7a974ceebdda0ef51be67220/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:02:47.696932 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09882e36-211f-4478-aff0-65dbc793c8d6 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:47.707541 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09882e36-211f-4478-aff0-65dbc793c8d6 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:47.763888 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb6ef69-8c46-4983-9357-abf31332309c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:47.764556 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6ef69-8c46-4983-9357-abf31332309c admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:47.764843 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6ef69-8c46-4983-9357-abf31332309c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:47.765150 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6ef69-8c46-4983-9357-abf31332309c admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:47.765889 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb6ef69-8c46-4983-9357-abf31332309c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:47.767528 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb6ef69-8c46-4983-9357-abf31332309c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:47.777154 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1392/2783] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:47.781935 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6374f-3974-4a40-8063-360ed6cd5c35 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:47.792591 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6374f-3974-4a40-8063-360ed6cd5c35 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:47.797471 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e986f83-6913-4b5b-b816-ee9a380e065e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:47.798337 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e986f83-6913-4b5b-b816-ee9a380e065e admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:47.798619 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e986f83-6913-4b5b-b816-ee9a380e065e admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:47.798891 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e986f83-6913-4b5b-b816-ee9a380e065e admin admin] PATH_INFO: `/v3/projects/4c9e146ca7b849a79a8ad989a215f922/users/94b4e11f7a974ceebdda0ef51be67220/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:47.800855 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e986f83-6913-4b5b-b816-ee9a380e065e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c9e146ca7b849a79a8ad989a215f922, user_id=94b4e11f7a974ceebdda0ef51be67220, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:47.803867 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e986f83-6913-4b5b-b816-ee9a380e065e admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:47.814459 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1392/2784] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 15:02:47 2022] PUT /identity/v3/projects/4c9e146ca7b849a79a8ad989a215f922/users/94b4e11f7a974ceebdda0ef51be67220/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:47.824248 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756014a-8ef5-4653-b85c-9387ba0e7168 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:47.824536 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756014a-8ef5-4653-b85c-9387ba0e7168 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:47.824814 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756014a-8ef5-4653-b85c-9387ba0e7168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:47.851482 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-f756014a-8ef5-4653-b85c-9387ba0e7168 None None] MFA Rules not processed for user `94b4e11f7a974ceebdda0ef51be67220`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:47.852708 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f756014a-8ef5-4653-b85c-9387ba0e7168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:47.915873 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:47.920146 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1393/2785] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 15:02:47 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:47.930868 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e986f83-6913-4b5b-b816-ee9a380e065e admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:47.932660 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-2e986f83-6913-4b5b-b816-ee9a380e065e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:47.990157 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e986f83-6913-4b5b-b816-ee9a380e065e admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:47.993668 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c384fba-6e10-4bf7-b8c4-a95146024bda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:47.994295 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c384fba-6e10-4bf7-b8c4-a95146024bda service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:47.994576 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c384fba-6e10-4bf7-b8c4-a95146024bda service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:47.994851 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c384fba-6e10-4bf7-b8c4-a95146024bda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:48.003589 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c384fba-6e10-4bf7-b8c4-a95146024bda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:48.005005 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c384fba-6e10-4bf7-b8c4-a95146024bda service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:48.018816 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1393/2786] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 15:02:47 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:02:48.160592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f756014a-8ef5-4653-b85c-9387ba0e7168 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:48.223997 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f756014a-8ef5-4653-b85c-9387ba0e7168 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:48.228637 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20eaaee-8eda-42c4-845d-c0830fb82449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:48.229516 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20eaaee-8eda-42c4-845d-c0830fb82449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:48.229803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20eaaee-8eda-42c4-845d-c0830fb82449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:48.230081 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20eaaee-8eda-42c4-845d-c0830fb82449 admin admin] PATH_INFO: `/v3/users/5182756ca21049d0b570337552292310` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:48.231917 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20eaaee-8eda-42c4-845d-c0830fb82449 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5182756ca21049d0b570337552292310)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:48.233850 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20eaaee-8eda-42c4-845d-c0830fb82449 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:48.288116 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e20eaaee-8eda-42c4-845d-c0830fb82449 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5182756ca21049d0b570337552292310'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:48.288778 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e20eaaee-8eda-42c4-845d-c0830fb82449 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5182756ca21049d0b570337552292310'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:48.302107 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e20eaaee-8eda-42c4-845d-c0830fb82449 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5182756ca21049d0b570337552292310'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:48.302767 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e20eaaee-8eda-42c4-845d-c0830fb82449 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5182756ca21049d0b570337552292310'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:48.310709 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e20eaaee-8eda-42c4-845d-c0830fb82449 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5182756ca21049d0b570337552292310'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:48.324368 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1394/2787] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 15:02:48 2022] DELETE /identity/v3/users/5182756ca21049d0b570337552292310 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:48.329496 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf037c-f02a-47b1-beb4-f3cf46ede7d9 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:48.329821 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf037c-f02a-47b1-beb4-f3cf46ede7d9 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:48.330107 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf037c-f02a-47b1-beb4-f3cf46ede7d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:48.348430 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-52cf037c-f02a-47b1-beb4-f3cf46ede7d9 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:48.349640 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-52cf037c-f02a-47b1-beb4-f3cf46ede7d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:48.411095 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1394/2788] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 15:02:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:48.420673 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20eaaee-8eda-42c4-845d-c0830fb82449 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:48.422377 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e20eaaee-8eda-42c4-845d-c0830fb82449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:48.480756 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20eaaee-8eda-42c4-845d-c0830fb82449 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:48.484517 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f1e7c-748b-4aa8-9eab-7896bf61134e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:48.485166 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f1e7c-748b-4aa8-9eab-7896bf61134e service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:48.485454 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f1e7c-748b-4aa8-9eab-7896bf61134e service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:48.485735 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f1e7c-748b-4aa8-9eab-7896bf61134e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:48.495457 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f1e7c-748b-4aa8-9eab-7896bf61134e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:48.496916 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f1e7c-748b-4aa8-9eab-7896bf61134e service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:48.506853 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1395/2789] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 15:02:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:02:48.782022 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf037c-f02a-47b1-beb4-f3cf46ede7d9 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:48.783784 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-52cf037c-f02a-47b1-beb4-f3cf46ede7d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:48.796445 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf037c-f02a-47b1-beb4-f3cf46ede7d9 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:48.801449 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da24a00-141e-4b87-96de-80c32b450935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:48.802521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da24a00-141e-4b87-96de-80c32b450935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:48.802802 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da24a00-141e-4b87-96de-80c32b450935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:48.803105 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da24a00-141e-4b87-96de-80c32b450935 admin admin] PATH_INFO: `/v3/projects/63bcfcfec16e409294be13f25820f8c6` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:48.804475 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da24a00-141e-4b87-96de-80c32b450935 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63bcfcfec16e409294be13f25820f8c6)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:48.806632 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da24a00-141e-4b87-96de-80c32b450935 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:48.857679 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-5da24a00-141e-4b87-96de-80c32b450935 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63bcfcfec16e409294be13f25820f8c6', 'request_id': 'req-5da24a00-141e-4b87-96de-80c32b450935'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:48.861885 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:48.868056 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-5da24a00-141e-4b87-96de-80c32b450935 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63bcfcfec16e409294be13f25820f8c6', 'request_id': 'req-5da24a00-141e-4b87-96de-80c32b450935'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:48.876241 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-5da24a00-141e-4b87-96de-80c32b450935 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63bcfcfec16e409294be13f25820f8c6', 'request_id': 'req-5da24a00-141e-4b87-96de-80c32b450935'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:48.887197 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-5da24a00-141e-4b87-96de-80c32b450935 admin admin] The token cache is being invalidate because project 63bcfcfec16e409294be13f25820f8c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:02:48.888106 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-5da24a00-141e-4b87-96de-80c32b450935 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:48.888767 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-5da24a00-141e-4b87-96de-80c32b450935 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:48.890314 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1395/2790] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 15:02:48 2022] DELETE /identity/v3/projects/63bcfcfec16e409294be13f25820f8c6 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:48.894622 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f1e7c-748b-4aa8-9eab-7896bf61134e service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:48.904230 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f1e7c-748b-4aa8-9eab-7896bf61134e service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:48.922787 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:48.968851 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efebc8dd-cf58-473d-b70f-eb244396f45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:48.969805 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efebc8dd-cf58-473d-b70f-eb244396f45a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:48.970566 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efebc8dd-cf58-473d-b70f-eb244396f45a admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:48.970850 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efebc8dd-cf58-473d-b70f-eb244396f45a admin admin] PATH_INFO: `/v3/users/94b4e11f7a974ceebdda0ef51be67220` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:48.972161 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efebc8dd-cf58-473d-b70f-eb244396f45a admin admin] RBAC: Authorizing `identity:delete_user(user_id=94b4e11f7a974ceebdda0ef51be67220)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:48.974147 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efebc8dd-cf58-473d-b70f-eb244396f45a admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:49.041173 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-efebc8dd-cf58-473d-b70f-eb244396f45a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94b4e11f7a974ceebdda0ef51be67220'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:49.042026 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-efebc8dd-cf58-473d-b70f-eb244396f45a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94b4e11f7a974ceebdda0ef51be67220'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:49.061607 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-efebc8dd-cf58-473d-b70f-eb244396f45a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94b4e11f7a974ceebdda0ef51be67220'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:49.062391 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-efebc8dd-cf58-473d-b70f-eb244396f45a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94b4e11f7a974ceebdda0ef51be67220'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:49.073332 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-efebc8dd-cf58-473d-b70f-eb244396f45a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94b4e11f7a974ceebdda0ef51be67220'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:49.092100 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1396/2791] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 15:02:48 2022] DELETE /identity/v3/users/94b4e11f7a974ceebdda0ef51be67220 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:49.657734 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da24a00-141e-4b87-96de-80c32b450935 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:49.659733 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-5da24a00-141e-4b87-96de-80c32b450935 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:49.673626 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da24a00-141e-4b87-96de-80c32b450935 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:49.737124 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1c617-087b-4ec5-806b-715049fbd853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:49.738003 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1c617-087b-4ec5-806b-715049fbd853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:49.738292 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1c617-087b-4ec5-806b-715049fbd853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:49.738570 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1c617-087b-4ec5-806b-715049fbd853 admin admin] PATH_INFO: `/v3/projects/4c9e146ca7b849a79a8ad989a215f922` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:49.739798 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea1c617-087b-4ec5-806b-715049fbd853 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c9e146ca7b849a79a8ad989a215f922)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:49.741710 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea1c617-087b-4ec5-806b-715049fbd853 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:49.789539 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-aea1c617-087b-4ec5-806b-715049fbd853 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c9e146ca7b849a79a8ad989a215f922', 'request_id': 'req-aea1c617-087b-4ec5-806b-715049fbd853'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:49.797881 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-aea1c617-087b-4ec5-806b-715049fbd853 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c9e146ca7b849a79a8ad989a215f922', 'request_id': 'req-aea1c617-087b-4ec5-806b-715049fbd853'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:49.804498 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-aea1c617-087b-4ec5-806b-715049fbd853 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c9e146ca7b849a79a8ad989a215f922', 'request_id': 'req-aea1c617-087b-4ec5-806b-715049fbd853'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:49.815483 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-aea1c617-087b-4ec5-806b-715049fbd853 admin admin] The token cache is being invalidate because project 4c9e146ca7b849a79a8ad989a215f922 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:02:49.815776 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-aea1c617-087b-4ec5-806b-715049fbd853 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:49.816422 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-aea1c617-087b-4ec5-806b-715049fbd853 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:49.817980 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1396/2792] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 15:02:49 2022] DELETE /identity/v3/projects/4c9e146ca7b849a79a8ad989a215f922 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:49.822899 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e000dfa-8b3c-41b2-991e-e9fc81dd92ea None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:49.823391 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e000dfa-8b3c-41b2-991e-e9fc81dd92ea None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:49.823501 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e000dfa-8b3c-41b2-991e-e9fc81dd92ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:49.841857 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-0e000dfa-8b3c-41b2-991e-e9fc81dd92ea None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:49.843079 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-0e000dfa-8b3c-41b2-991e-e9fc81dd92ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:49.904726 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1397/2793] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 15:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:49.913824 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859e672-b3c4-4f69-8d99-b3d168cda228 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:49.914115 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859e672-b3c4-4f69-8d99-b3d168cda228 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:49.914396 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859e672-b3c4-4f69-8d99-b3d168cda228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:49.934002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-7859e672-b3c4-4f69-8d99-b3d168cda228 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:49.936208 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-7859e672-b3c4-4f69-8d99-b3d168cda228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:49.944860 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1397/2794] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 15:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:49.949030 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e000dfa-8b3c-41b2-991e-e9fc81dd92ea None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:49.959130 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e000dfa-8b3c-41b2-991e-e9fc81dd92ea None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:49.963793 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540c000-761f-4bdb-8a56-9a618d113461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:49.964454 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540c000-761f-4bdb-8a56-9a618d113461 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:49.964745 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540c000-761f-4bdb-8a56-9a618d113461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:49.965018 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540c000-761f-4bdb-8a56-9a618d113461 admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:49.965781 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540c000-761f-4bdb-8a56-9a618d113461 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:02:49.966090 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540c000-761f-4bdb-8a56-9a618d113461 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:49.967729 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540c000-761f-4bdb-8a56-9a618d113461 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:49.979388 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1398/2795] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 15:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:02:49.984327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7859e672-b3c4-4f69-8d99-b3d168cda228 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:49.995919 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7859e672-b3c4-4f69-8d99-b3d168cda228 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:50.001258 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaea0fa-e698-4922-b8cf-62bb843378ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:50.001919 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaea0fa-e698-4922-b8cf-62bb843378ea admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:50.002207 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaea0fa-e698-4922-b8cf-62bb843378ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:50.002486 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaea0fa-e698-4922-b8cf-62bb843378ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:50.003240 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaea0fa-e698-4922-b8cf-62bb843378ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:50.005176 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaea0fa-e698-4922-b8cf-62bb843378ea admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:50.035907 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1398/2796] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 15:02:49 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:50.037025 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540c000-761f-4bdb-8a56-9a618d113461 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:50.052021 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540c000-761f-4bdb-8a56-9a618d113461 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:50.080908 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:50.117544 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3919469c-a199-4d7f-a48a-1cd66394d618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:50.118438 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919469c-a199-4d7f-a48a-1cd66394d618 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:50.118554 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919469c-a199-4d7f-a48a-1cd66394d618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:50.118825 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919469c-a199-4d7f-a48a-1cd66394d618 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:50.119570 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3919469c-a199-4d7f-a48a-1cd66394d618 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:50.121529 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3919469c-a199-4d7f-a48a-1cd66394d618 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:50.145494 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1399/2797] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 15:02:50 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:02:50.146852 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaea0fa-e698-4922-b8cf-62bb843378ea admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:50.158459 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaea0fa-e698-4922-b8cf-62bb843378ea admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:50.163118 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7416a6-edb5-4cb0-a1df-46f7a2063a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:50.163785 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7416a6-edb5-4cb0-a1df-46f7a2063a24 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:50.164070 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7416a6-edb5-4cb0-a1df-46f7a2063a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:50.164347 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7416a6-edb5-4cb0-a1df-46f7a2063a24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:50.165106 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7416a6-edb5-4cb0-a1df-46f7a2063a24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:50.166693 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7416a6-edb5-4cb0-a1df-46f7a2063a24 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:50.178873 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1399/2798] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:50.183282 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3919469c-a199-4d7f-a48a-1cd66394d618 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:50.194357 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3919469c-a199-4d7f-a48a-1cd66394d618 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:50.199359 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd1123-9d00-433a-b5e9-9e82a592b01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:50.200146 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd1123-9d00-433a-b5e9-9e82a592b01e admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:50.200438 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd1123-9d00-433a-b5e9-9e82a592b01e admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:50.200724 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd1123-9d00-433a-b5e9-9e82a592b01e admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:50.201481 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fd1123-9d00-433a-b5e9-9e82a592b01e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:50.203129 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fd1123-9d00-433a-b5e9-9e82a592b01e admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:50.214094 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1400/2799] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:50.218830 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7416a6-edb5-4cb0-a1df-46f7a2063a24 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:50.228903 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7416a6-edb5-4cb0-a1df-46f7a2063a24 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:50.234173 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d96caf4-880c-4ea4-a580-5e3f94daa71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:50.235101 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96caf4-880c-4ea4-a580-5e3f94daa71c admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:50.235416 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96caf4-880c-4ea4-a580-5e3f94daa71c admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:50.235722 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96caf4-880c-4ea4-a580-5e3f94daa71c admin admin] PATH_INFO: `/v3/projects/dbafa0d4f81b4d8398ba48502f70f1a8/users/a9050ae0078545c5a926a436407d2322/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:50.251084 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d96caf4-880c-4ea4-a580-5e3f94daa71c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbafa0d4f81b4d8398ba48502f70f1a8, user_id=a9050ae0078545c5a926a436407d2322, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:50.253729 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d96caf4-880c-4ea4-a580-5e3f94daa71c admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:50.274148 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1400/2800] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 15:02:50 2022] PUT /identity/v3/projects/dbafa0d4f81b4d8398ba48502f70f1a8/users/a9050ae0078545c5a926a436407d2322/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:50.284886 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219b325-c63c-4fa3-b12e-dd43f897c211 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:50.285177 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219b325-c63c-4fa3-b12e-dd43f897c211 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:50.285481 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219b325-c63c-4fa3-b12e-dd43f897c211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:50.312649 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-f219b325-c63c-4fa3-b12e-dd43f897c211 None None] MFA Rules not processed for user `a9050ae0078545c5a926a436407d2322`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:50.313856 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f219b325-c63c-4fa3-b12e-dd43f897c211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:50.380606 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1401/2801] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 15:02:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:02:50.385109 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d96caf4-880c-4ea4-a580-5e3f94daa71c admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:50.394046 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:50.458524 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d96caf4-880c-4ea4-a580-5e3f94daa71c admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:50.463194 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6f2ca-4e0a-4200-92d6-168e6e1fee10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:50.463863 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6f2ca-4e0a-4200-92d6-168e6e1fee10 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:50.464147 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6f2ca-4e0a-4200-92d6-168e6e1fee10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:50.464431 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6f2ca-4e0a-4200-92d6-168e6e1fee10 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:50.465128 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f6f2ca-4e0a-4200-92d6-168e6e1fee10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:50.467665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f6f2ca-4e0a-4200-92d6-168e6e1fee10 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:50.497995 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1401/2802] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 15:02:50 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:02:50.499173 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f219b325-c63c-4fa3-b12e-dd43f897c211 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:50.511550 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f219b325-c63c-4fa3-b12e-dd43f897c211 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:50.573763 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f4cfc-d358-417e-90bc-001f9755de83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:50.574449 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f4cfc-d358-417e-90bc-001f9755de83 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:50.574731 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f4cfc-d358-417e-90bc-001f9755de83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:50.575641 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f4cfc-d358-417e-90bc-001f9755de83 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:50.576321 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366f4cfc-d358-417e-90bc-001f9755de83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:50.578291 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366f4cfc-d358-417e-90bc-001f9755de83 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:50.604280 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1402/2803] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 15:02:50 2022] POST /identity/v3/users => generated 300 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:50.605758 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6f2ca-4e0a-4200-92d6-168e6e1fee10 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:50.617081 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6f2ca-4e0a-4200-92d6-168e6e1fee10 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:50.621788 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c0a4f-c05d-4b83-8922-5ebb7b365d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:50.622469 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c0a4f-c05d-4b83-8922-5ebb7b365d1f admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:50.622750 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c0a4f-c05d-4b83-8922-5ebb7b365d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:50.623051 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c0a4f-c05d-4b83-8922-5ebb7b365d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:50.626191 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c0a4f-c05d-4b83-8922-5ebb7b365d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:50.626191 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c0a4f-c05d-4b83-8922-5ebb7b365d1f admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:50.644517 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1402/2804] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:50.645618 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f4cfc-d358-417e-90bc-001f9755de83 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:50.657049 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f4cfc-d358-417e-90bc-001f9755de83 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:50.662560 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a82778-068e-4191-9911-ba2f464cd1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:50.663618 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a82778-068e-4191-9911-ba2f464cd1e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:50.663966 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a82778-068e-4191-9911-ba2f464cd1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:50.664307 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a82778-068e-4191-9911-ba2f464cd1e9 admin admin] PATH_INFO: `/v3/projects/013cac84cbe744428894a2c5bf031680/users/ebd356ecfcb04697a794bf2cc8959555/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:50.681257 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a82778-068e-4191-9911-ba2f464cd1e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=013cac84cbe744428894a2c5bf031680, user_id=ebd356ecfcb04697a794bf2cc8959555, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:50.684093 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a82778-068e-4191-9911-ba2f464cd1e9 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:50.700406 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1403/2805] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 15:02:50 2022] PUT /identity/v3/projects/013cac84cbe744428894a2c5bf031680/users/ebd356ecfcb04697a794bf2cc8959555/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:50.704404 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c0a4f-c05d-4b83-8922-5ebb7b365d1f admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:50.714811 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c0a4f-c05d-4b83-8922-5ebb7b365d1f admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:50.777533 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3585a3e-5172-4acf-857d-5a68b5621925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:50.778249 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3585a3e-5172-4acf-857d-5a68b5621925 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:50.778538 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3585a3e-5172-4acf-857d-5a68b5621925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:50.778816 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3585a3e-5172-4acf-857d-5a68b5621925 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:50.779592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3585a3e-5172-4acf-857d-5a68b5621925 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:50.781246 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3585a3e-5172-4acf-857d-5a68b5621925 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:50.791961 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1403/2806] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:50.796876 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a82778-068e-4191-9911-ba2f464cd1e9 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:50.802536 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:50.809163 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a82778-068e-4191-9911-ba2f464cd1e9 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:50.815136 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec3fd9d-764b-4730-ba62-0e307cc00516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:50.816044 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3fd9d-764b-4730-ba62-0e307cc00516 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:50.816333 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3fd9d-764b-4730-ba62-0e307cc00516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:50.816612 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3fd9d-764b-4730-ba62-0e307cc00516 admin admin] PATH_INFO: `/v3/projects/013cac84cbe744428894a2c5bf031680/users/ebd356ecfcb04697a794bf2cc8959555/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:50.818622 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec3fd9d-764b-4730-ba62-0e307cc00516 admin admin] RBAC: Authorizing `identity:create_grant(project_id=013cac84cbe744428894a2c5bf031680, user_id=ebd356ecfcb04697a794bf2cc8959555, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:50.822411 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec3fd9d-764b-4730-ba62-0e307cc00516 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:50.833855 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1404/2807] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 15:02:50 2022] PUT /identity/v3/projects/013cac84cbe744428894a2c5bf031680/users/ebd356ecfcb04697a794bf2cc8959555/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:50.845344 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a4dcf-23c1-4315-b75f-f31e21fe4ebc None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:50.845636 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a4dcf-23c1-4315-b75f-f31e21fe4ebc None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:50.845919 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a4dcf-23c1-4315-b75f-f31e21fe4ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:50.881634 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-876a4dcf-23c1-4315-b75f-f31e21fe4ebc None None] MFA Rules not processed for user `ebd356ecfcb04697a794bf2cc8959555`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:50.885458 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-876a4dcf-23c1-4315-b75f-f31e21fe4ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:50.987153 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1404/2808] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 15:02:50 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:50.995261 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec3fd9d-764b-4730-ba62-0e307cc00516 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:50.997148 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-7ec3fd9d-764b-4730-ba62-0e307cc00516 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:51.065955 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec3fd9d-764b-4730-ba62-0e307cc00516 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:51.069864 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9420e1-51b0-49fd-bab3-1fda68456c33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:51.070509 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9420e1-51b0-49fd-bab3-1fda68456c33 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:51.070788 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9420e1-51b0-49fd-bab3-1fda68456c33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:51.071099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9420e1-51b0-49fd-bab3-1fda68456c33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:51.080306 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9420e1-51b0-49fd-bab3-1fda68456c33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:51.081742 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9420e1-51b0-49fd-bab3-1fda68456c33 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:51.091978 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1405/2809] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 15:02:50 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:02:51.155279 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a4dcf-23c1-4315-b75f-f31e21fe4ebc None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:51.159576 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:51.223015 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a4dcf-23c1-4315-b75f-f31e21fe4ebc None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:51.227592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04040e0d-1028-43da-afee-305b533a73f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:51.228465 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04040e0d-1028-43da-afee-305b533a73f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:51.228752 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04040e0d-1028-43da-afee-305b533a73f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:51.229027 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04040e0d-1028-43da-afee-305b533a73f5 admin admin] PATH_INFO: `/v3/users/a9050ae0078545c5a926a436407d2322` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:51.230258 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04040e0d-1028-43da-afee-305b533a73f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9050ae0078545c5a926a436407d2322)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:51.232907 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04040e0d-1028-43da-afee-305b533a73f5 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:51.283743 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-04040e0d-1028-43da-afee-305b533a73f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9050ae0078545c5a926a436407d2322'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:51.284401 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-04040e0d-1028-43da-afee-305b533a73f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9050ae0078545c5a926a436407d2322'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:51.296631 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-04040e0d-1028-43da-afee-305b533a73f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9050ae0078545c5a926a436407d2322'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:51.304648 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-04040e0d-1028-43da-afee-305b533a73f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9050ae0078545c5a926a436407d2322'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:51.313103 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-04040e0d-1028-43da-afee-305b533a73f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9050ae0078545c5a926a436407d2322'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:51.321037 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1405/2810] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 15:02:51 2022] DELETE /identity/v3/users/a9050ae0078545c5a926a436407d2322 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:51.326497 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356486d4-c123-4be2-bac7-bba37e9b5bfc None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:51.326797 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356486d4-c123-4be2-bac7-bba37e9b5bfc None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:51.327106 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356486d4-c123-4be2-bac7-bba37e9b5bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:51.345804 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-356486d4-c123-4be2-bac7-bba37e9b5bfc None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:51.347056 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-356486d4-c123-4be2-bac7-bba37e9b5bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:51.408830 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1406/2811] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 15:02:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:02:51.418376 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04040e0d-1028-43da-afee-305b533a73f5 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:51.419895 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-04040e0d-1028-43da-afee-305b533a73f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:51.489856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04040e0d-1028-43da-afee-305b533a73f5 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:51.495374 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb70bc4-c0e3-437d-b8fd-c05fb25529f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:51.496120 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb70bc4-c0e3-437d-b8fd-c05fb25529f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:51.496469 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb70bc4-c0e3-437d-b8fd-c05fb25529f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:51.497036 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb70bc4-c0e3-437d-b8fd-c05fb25529f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:51.507891 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb70bc4-c0e3-437d-b8fd-c05fb25529f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:51.509514 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb70bc4-c0e3-437d-b8fd-c05fb25529f6 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:51.526346 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1406/2812] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 15:02:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:02:51.800173 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356486d4-c123-4be2-bac7-bba37e9b5bfc None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:51.801878 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-356486d4-c123-4be2-bac7-bba37e9b5bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:51.816494 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356486d4-c123-4be2-bac7-bba37e9b5bfc None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:51.821426 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37885e68-1101-445c-9230-fdf250131354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:51.822444 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37885e68-1101-445c-9230-fdf250131354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:51.822811 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37885e68-1101-445c-9230-fdf250131354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:51.823295 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37885e68-1101-445c-9230-fdf250131354 admin admin] PATH_INFO: `/v3/projects/dbafa0d4f81b4d8398ba48502f70f1a8` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:51.824844 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37885e68-1101-445c-9230-fdf250131354 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbafa0d4f81b4d8398ba48502f70f1a8)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:51.832370 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37885e68-1101-445c-9230-fdf250131354 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:51.884641 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-37885e68-1101-445c-9230-fdf250131354 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbafa0d4f81b4d8398ba48502f70f1a8', 'request_id': 'req-37885e68-1101-445c-9230-fdf250131354'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:51.894146 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-37885e68-1101-445c-9230-fdf250131354 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbafa0d4f81b4d8398ba48502f70f1a8', 'request_id': 'req-37885e68-1101-445c-9230-fdf250131354'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:51.902392 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-37885e68-1101-445c-9230-fdf250131354 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbafa0d4f81b4d8398ba48502f70f1a8', 'request_id': 'req-37885e68-1101-445c-9230-fdf250131354'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:51.915270 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-37885e68-1101-445c-9230-fdf250131354 admin admin] The token cache is being invalidate because project dbafa0d4f81b4d8398ba48502f70f1a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:02:51.915565 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-37885e68-1101-445c-9230-fdf250131354 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:51.916227 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-37885e68-1101-445c-9230-fdf250131354 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:51.917842 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1407/2813] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 15:02:51 2022] DELETE /identity/v3/projects/dbafa0d4f81b4d8398ba48502f70f1a8 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:51.922365 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb70bc4-c0e3-437d-b8fd-c05fb25529f6 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:51.936066 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb70bc4-c0e3-437d-b8fd-c05fb25529f6 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:51.960046 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:52.000840 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1870b7-7212-4aaa-91c7-e7aa5c784299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:52.001717 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1870b7-7212-4aaa-91c7-e7aa5c784299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:52.002002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1870b7-7212-4aaa-91c7-e7aa5c784299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:52.002278 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1870b7-7212-4aaa-91c7-e7aa5c784299 admin admin] PATH_INFO: `/v3/users/ebd356ecfcb04697a794bf2cc8959555` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:52.003509 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1870b7-7212-4aaa-91c7-e7aa5c784299 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebd356ecfcb04697a794bf2cc8959555)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:52.005446 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1870b7-7212-4aaa-91c7-e7aa5c784299 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:52.054929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6f1870b7-7212-4aaa-91c7-e7aa5c784299 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebd356ecfcb04697a794bf2cc8959555'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:52.055620 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6f1870b7-7212-4aaa-91c7-e7aa5c784299 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebd356ecfcb04697a794bf2cc8959555'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:52.067849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6f1870b7-7212-4aaa-91c7-e7aa5c784299 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebd356ecfcb04697a794bf2cc8959555'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:52.075018 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6f1870b7-7212-4aaa-91c7-e7aa5c784299 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebd356ecfcb04697a794bf2cc8959555'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:52.081770 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6f1870b7-7212-4aaa-91c7-e7aa5c784299 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebd356ecfcb04697a794bf2cc8959555'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:52.087601 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1407/2814] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 15:02:51 2022] DELETE /identity/v3/users/ebd356ecfcb04697a794bf2cc8959555 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:52.367571 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37885e68-1101-445c-9230-fdf250131354 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:52.369061 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-37885e68-1101-445c-9230-fdf250131354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:52.387053 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37885e68-1101-445c-9230-fdf250131354 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:52.401158 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:52.447880 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a7fb2-4d34-4884-845a-d515bc690bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:52.448670 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a7fb2-4d34-4884-845a-d515bc690bd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:52.448956 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a7fb2-4d34-4884-845a-d515bc690bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:52.449235 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a7fb2-4d34-4884-845a-d515bc690bd2 admin admin] PATH_INFO: `/v3/projects/013cac84cbe744428894a2c5bf031680` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:52.450977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2a7fb2-4d34-4884-845a-d515bc690bd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=013cac84cbe744428894a2c5bf031680)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:52.452921 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2a7fb2-4d34-4884-845a-d515bc690bd2 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:52.497744 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ab2a7fb2-4d34-4884-845a-d515bc690bd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '013cac84cbe744428894a2c5bf031680', 'request_id': 'req-ab2a7fb2-4d34-4884-845a-d515bc690bd2'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:52.504623 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ab2a7fb2-4d34-4884-845a-d515bc690bd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '013cac84cbe744428894a2c5bf031680', 'request_id': 'req-ab2a7fb2-4d34-4884-845a-d515bc690bd2'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:52.510619 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ab2a7fb2-4d34-4884-845a-d515bc690bd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '013cac84cbe744428894a2c5bf031680', 'request_id': 'req-ab2a7fb2-4d34-4884-845a-d515bc690bd2'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:52.521261 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ab2a7fb2-4d34-4884-845a-d515bc690bd2 admin admin] The token cache is being invalidate because project 013cac84cbe744428894a2c5bf031680 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:02:52.521551 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ab2a7fb2-4d34-4884-845a-d515bc690bd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:52.522182 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ab2a7fb2-4d34-4884-845a-d515bc690bd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:52.523747 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1408/2815] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 15:02:52 2022] DELETE /identity/v3/projects/013cac84cbe744428894a2c5bf031680 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:52.529627 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258590b3-05a1-47cd-9abd-c071c32563e0 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:52.529926 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258590b3-05a1-47cd-9abd-c071c32563e0 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:52.530212 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258590b3-05a1-47cd-9abd-c071c32563e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:52.548358 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-258590b3-05a1-47cd-9abd-c071c32563e0 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:52.549665 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-258590b3-05a1-47cd-9abd-c071c32563e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:52.608242 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1408/2816] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 15:02:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:52.617071 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28471b17-7d11-45a9-9a82-30920739306c None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:52.617365 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28471b17-7d11-45a9-9a82-30920739306c None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:52.617644 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28471b17-7d11-45a9-9a82-30920739306c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:52.635886 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-28471b17-7d11-45a9-9a82-30920739306c None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:52.637070 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-28471b17-7d11-45a9-9a82-30920739306c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:52.645547 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1409/2817] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 15:02:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:52.649705 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258590b3-05a1-47cd-9abd-c071c32563e0 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:52.659681 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258590b3-05a1-47cd-9abd-c071c32563e0 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:52.664268 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0bdfb-1250-4574-b21c-cfeb2d136e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:52.664924 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0bdfb-1250-4574-b21c-cfeb2d136e64 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:52.665209 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0bdfb-1250-4574-b21c-cfeb2d136e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:52.665485 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0bdfb-1250-4574-b21c-cfeb2d136e64 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:52.666207 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa0bdfb-1250-4574-b21c-cfeb2d136e64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:02:52.666517 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa0bdfb-1250-4574-b21c-cfeb2d136e64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:52.668149 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa0bdfb-1250-4574-b21c-cfeb2d136e64 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:52.679844 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1409/2818] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 15:02:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:02:52.684143 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28471b17-7d11-45a9-9a82-30920739306c None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:52.693999 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28471b17-7d11-45a9-9a82-30920739306c None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:52.698658 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc79b4-103d-48bd-8f36-03aae7767632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:52.699279 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc79b4-103d-48bd-8f36-03aae7767632 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:52.699564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc79b4-103d-48bd-8f36-03aae7767632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:52.699860 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc79b4-103d-48bd-8f36-03aae7767632 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:52.701121 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dc79b4-103d-48bd-8f36-03aae7767632 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:52.703108 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dc79b4-103d-48bd-8f36-03aae7767632 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:52.747547 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1410/2819] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 15:02:52 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:52.752563 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0bdfb-1250-4574-b21c-cfeb2d136e64 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:52.764464 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0bdfb-1250-4574-b21c-cfeb2d136e64 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:52.794219 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:52.823970 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7931e7-ba48-4210-aa8d-aa936a15215b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:52.824633 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7931e7-ba48-4210-aa8d-aa936a15215b admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:52.824919 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7931e7-ba48-4210-aa8d-aa936a15215b admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:52.825194 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7931e7-ba48-4210-aa8d-aa936a15215b admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:52.825877 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7931e7-ba48-4210-aa8d-aa936a15215b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:52.827817 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7931e7-ba48-4210-aa8d-aa936a15215b admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:52.846212 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1410/2820] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 15:02:52 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:52.850516 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc79b4-103d-48bd-8f36-03aae7767632 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:52.861456 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc79b4-103d-48bd-8f36-03aae7767632 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:52.866197 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a7d02-cf93-4fbd-923c-7f44a2621768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:52.866846 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a7d02-cf93-4fbd-923c-7f44a2621768 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:52.867167 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a7d02-cf93-4fbd-923c-7f44a2621768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:52.867447 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a7d02-cf93-4fbd-923c-7f44a2621768 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:52.868174 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351a7d02-cf93-4fbd-923c-7f44a2621768 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:52.869774 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351a7d02-cf93-4fbd-923c-7f44a2621768 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:52.880587 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1411/2821] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:52.884302 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7931e7-ba48-4210-aa8d-aa936a15215b admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:52.894232 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7931e7-ba48-4210-aa8d-aa936a15215b admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:52.898904 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d315f035-f169-424b-ad94-6a1e533b0d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:52.899589 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315f035-f169-424b-ad94-6a1e533b0d98 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:52.899873 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315f035-f169-424b-ad94-6a1e533b0d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:52.900149 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315f035-f169-424b-ad94-6a1e533b0d98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:52.900865 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d315f035-f169-424b-ad94-6a1e533b0d98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:52.902443 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d315f035-f169-424b-ad94-6a1e533b0d98 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:52.912170 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1411/2822] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:52.916109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a7d02-cf93-4fbd-923c-7f44a2621768 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:52.926909 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a7d02-cf93-4fbd-923c-7f44a2621768 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:52.931889 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06e779-a19d-49c9-bcf9-ffd62cd6a324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:52.932770 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06e779-a19d-49c9-bcf9-ffd62cd6a324 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:52.933055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06e779-a19d-49c9-bcf9-ffd62cd6a324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:52.933328 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06e779-a19d-49c9-bcf9-ffd62cd6a324 admin admin] PATH_INFO: `/v3/projects/a47832d6f45f4805b784d19451773188/users/28bc97e878614fb2af1300f14123ee14/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:52.948118 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be06e779-a19d-49c9-bcf9-ffd62cd6a324 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a47832d6f45f4805b784d19451773188, user_id=28bc97e878614fb2af1300f14123ee14, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:52.950792 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be06e779-a19d-49c9-bcf9-ffd62cd6a324 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:52.963313 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1412/2823] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 15:02:52 2022] PUT /identity/v3/projects/a47832d6f45f4805b784d19451773188/users/28bc97e878614fb2af1300f14123ee14/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:52.973247 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92d0ed-0300-49d3-94a2-2a072d07e2f4 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:52.973707 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92d0ed-0300-49d3-94a2-2a072d07e2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:52.973832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92d0ed-0300-49d3-94a2-2a072d07e2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:53.000849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-8b92d0ed-0300-49d3-94a2-2a072d07e2f4 None None] MFA Rules not processed for user `28bc97e878614fb2af1300f14123ee14`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:53.002032 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-8b92d0ed-0300-49d3-94a2-2a072d07e2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:53.064805 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1412/2824] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 15:02:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:53.069158 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06e779-a19d-49c9-bcf9-ffd62cd6a324 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:53.120527 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:53.139950 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06e779-a19d-49c9-bcf9-ffd62cd6a324 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:53.144853 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a8dd6-60fd-4733-8b6a-6d18c3a3aec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:53.145650 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a8dd6-60fd-4733-8b6a-6d18c3a3aec8 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:53.145969 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a8dd6-60fd-4733-8b6a-6d18c3a3aec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:53.146295 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a8dd6-60fd-4733-8b6a-6d18c3a3aec8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:53.147028 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a8dd6-60fd-4733-8b6a-6d18c3a3aec8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:53.149315 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a8dd6-60fd-4733-8b6a-6d18c3a3aec8 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:53.178484 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1413/2825] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 15:02:53 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:53.182242 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92d0ed-0300-49d3-94a2-2a072d07e2f4 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:53.193270 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92d0ed-0300-49d3-94a2-2a072d07e2f4 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:53.250986 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c5be48-b21a-439b-ac34-572ba4817783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:53.251617 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5be48-b21a-439b-ac34-572ba4817783 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:53.251909 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5be48-b21a-439b-ac34-572ba4817783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:53.252189 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5be48-b21a-439b-ac34-572ba4817783 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:53.252872 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c5be48-b21a-439b-ac34-572ba4817783 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:53.254812 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c5be48-b21a-439b-ac34-572ba4817783 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:53.274017 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1413/2826] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 15:02:53 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:53.279497 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a8dd6-60fd-4733-8b6a-6d18c3a3aec8 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:53.289682 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a8dd6-60fd-4733-8b6a-6d18c3a3aec8 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:53.295034 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b646f-73c6-4d5b-b94a-3948b7dcc01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:53.295681 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b646f-73c6-4d5b-b94a-3948b7dcc01c admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:53.295979 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b646f-73c6-4d5b-b94a-3948b7dcc01c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:53.296498 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b646f-73c6-4d5b-b94a-3948b7dcc01c admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:53.297230 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446b646f-73c6-4d5b-b94a-3948b7dcc01c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:53.298820 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446b646f-73c6-4d5b-b94a-3948b7dcc01c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:53.309500 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1414/2827] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:53.313555 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c5be48-b21a-439b-ac34-572ba4817783 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:53.323521 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c5be48-b21a-439b-ac34-572ba4817783 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:53.328282 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8375a9-6f3a-4ada-aeb6-249f398817ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:53.329181 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8375a9-6f3a-4ada-aeb6-249f398817ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:53.329489 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8375a9-6f3a-4ada-aeb6-249f398817ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:53.329783 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8375a9-6f3a-4ada-aeb6-249f398817ed admin admin] PATH_INFO: `/v3/projects/76b82c295eb04443b2bc0595901ab41c/users/4e19f8285b574f17b63a09660fb50785/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:53.345143 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8375a9-6f3a-4ada-aeb6-249f398817ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=76b82c295eb04443b2bc0595901ab41c, user_id=4e19f8285b574f17b63a09660fb50785, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:53.347863 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8375a9-6f3a-4ada-aeb6-249f398817ed admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:53.361753 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1414/2828] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 15:02:53 2022] PUT /identity/v3/projects/76b82c295eb04443b2bc0595901ab41c/users/4e19f8285b574f17b63a09660fb50785/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:53.365601 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b646f-73c6-4d5b-b94a-3948b7dcc01c admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:53.376004 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b646f-73c6-4d5b-b94a-3948b7dcc01c admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:53.436781 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc1a60-3218-4e1e-83e9-c3b232427188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:53.437473 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc1a60-3218-4e1e-83e9-c3b232427188 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:53.437771 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc1a60-3218-4e1e-83e9-c3b232427188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:53.438051 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc1a60-3218-4e1e-83e9-c3b232427188 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:53.438787 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc1a60-3218-4e1e-83e9-c3b232427188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:53.440572 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc1a60-3218-4e1e-83e9-c3b232427188 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:53.450887 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1415/2829] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:53.455662 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8375a9-6f3a-4ada-aeb6-249f398817ed admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:53.462247 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:53.466986 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8375a9-6f3a-4ada-aeb6-249f398817ed admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:53.472760 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e11965-9a1a-4db5-8664-18d982788fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:53.473645 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e11965-9a1a-4db5-8664-18d982788fda admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:53.473932 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e11965-9a1a-4db5-8664-18d982788fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:53.474217 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e11965-9a1a-4db5-8664-18d982788fda admin admin] PATH_INFO: `/v3/projects/76b82c295eb04443b2bc0595901ab41c/users/4e19f8285b574f17b63a09660fb50785/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:53.476243 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e11965-9a1a-4db5-8664-18d982788fda admin admin] RBAC: Authorizing `identity:create_grant(project_id=76b82c295eb04443b2bc0595901ab41c, user_id=4e19f8285b574f17b63a09660fb50785, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:53.478881 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e11965-9a1a-4db5-8664-18d982788fda admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:53.489622 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1415/2830] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 15:02:53 2022] PUT /identity/v3/projects/76b82c295eb04443b2bc0595901ab41c/users/4e19f8285b574f17b63a09660fb50785/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:53.500736 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510dfac0-06ae-4c2a-acbb-a3c19821131e None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:53.501101 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510dfac0-06ae-4c2a-acbb-a3c19821131e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:53.501315 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510dfac0-06ae-4c2a-acbb-a3c19821131e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:53.527423 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-510dfac0-06ae-4c2a-acbb-a3c19821131e None None] MFA Rules not processed for user `4e19f8285b574f17b63a09660fb50785`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:53.529210 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-510dfac0-06ae-4c2a-acbb-a3c19821131e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:53.596153 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1416/2831] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 15:02:53 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:53.606595 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e11965-9a1a-4db5-8664-18d982788fda admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:53.608336 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-75e11965-9a1a-4db5-8664-18d982788fda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:53.666593 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e11965-9a1a-4db5-8664-18d982788fda admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:53.670400 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3effb2d7-b4d0-4da7-a949-ac7140e26b08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:53.671331 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3effb2d7-b4d0-4da7-a949-ac7140e26b08 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:53.671620 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3effb2d7-b4d0-4da7-a949-ac7140e26b08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:53.671901 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3effb2d7-b4d0-4da7-a949-ac7140e26b08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:53.729016 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3effb2d7-b4d0-4da7-a949-ac7140e26b08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:53.730476 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3effb2d7-b4d0-4da7-a949-ac7140e26b08 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:53.739672 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1416/2832] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 15:02:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:02:56.252292 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510dfac0-06ae-4c2a-acbb-a3c19821131e None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:56.310652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:56.345705 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510dfac0-06ae-4c2a-acbb-a3c19821131e None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:56.350654 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d11872-20a6-4984-b70d-587e68c23724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:56.351724 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d11872-20a6-4984-b70d-587e68c23724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:56.354012 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d11872-20a6-4984-b70d-587e68c23724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:56.354012 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d11872-20a6-4984-b70d-587e68c23724 admin admin] PATH_INFO: `/v3/users/28bc97e878614fb2af1300f14123ee14` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:56.354012 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d11872-20a6-4984-b70d-587e68c23724 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28bc97e878614fb2af1300f14123ee14)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:56.356055 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d11872-20a6-4984-b70d-587e68c23724 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:56.416912 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-33d11872-20a6-4984-b70d-587e68c23724 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28bc97e878614fb2af1300f14123ee14'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:56.417607 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-33d11872-20a6-4984-b70d-587e68c23724 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28bc97e878614fb2af1300f14123ee14'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:56.434293 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-33d11872-20a6-4984-b70d-587e68c23724 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28bc97e878614fb2af1300f14123ee14'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:56.435076 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-33d11872-20a6-4984-b70d-587e68c23724 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28bc97e878614fb2af1300f14123ee14'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:56.447623 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-33d11872-20a6-4984-b70d-587e68c23724 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28bc97e878614fb2af1300f14123ee14'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:56.464820 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1417/2833] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 15:02:56 2022] DELETE /identity/v3/users/28bc97e878614fb2af1300f14123ee14 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:56.469759 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ef487-e43e-4fbe-b408-70a8f0012c49 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:56.470274 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ef487-e43e-4fbe-b408-70a8f0012c49 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:56.470480 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ef487-e43e-4fbe-b408-70a8f0012c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:56.488815 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-914ef487-e43e-4fbe-b408-70a8f0012c49 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:56.490045 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-914ef487-e43e-4fbe-b408-70a8f0012c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:56.554954 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1417/2834] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 15:02:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:56.564530 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d11872-20a6-4984-b70d-587e68c23724 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:56.565933 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-33d11872-20a6-4984-b70d-587e68c23724 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:56.624599 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d11872-20a6-4984-b70d-587e68c23724 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:56.628242 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934e640b-d1a9-4b10-8b70-5295779e609c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:56.628892 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934e640b-d1a9-4b10-8b70-5295779e609c service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:56.629175 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934e640b-d1a9-4b10-8b70-5295779e609c service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:56.629491 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934e640b-d1a9-4b10-8b70-5295779e609c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:56.639436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934e640b-d1a9-4b10-8b70-5295779e609c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:56.641013 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934e640b-d1a9-4b10-8b70-5295779e609c service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:56.651028 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1418/2835] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 15:02:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:02:56.923237 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ef487-e43e-4fbe-b408-70a8f0012c49 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:56.924774 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-914ef487-e43e-4fbe-b408-70a8f0012c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:56.939926 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ef487-e43e-4fbe-b408-70a8f0012c49 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:56.944802 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2d10c-ba59-4699-957d-7161e20f4d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:56.945763 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2d10c-ba59-4699-957d-7161e20f4d76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:56.946129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2d10c-ba59-4699-957d-7161e20f4d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:56.947240 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2d10c-ba59-4699-957d-7161e20f4d76 admin admin] PATH_INFO: `/v3/projects/a47832d6f45f4805b784d19451773188` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:56.948711 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a2d10c-ba59-4699-957d-7161e20f4d76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a47832d6f45f4805b784d19451773188)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:56.952174 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a2d10c-ba59-4699-957d-7161e20f4d76 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:56.996792 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:57.004644 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-64a2d10c-ba59-4699-957d-7161e20f4d76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a47832d6f45f4805b784d19451773188', 'request_id': 'req-64a2d10c-ba59-4699-957d-7161e20f4d76'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:57.014801 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-64a2d10c-ba59-4699-957d-7161e20f4d76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a47832d6f45f4805b784d19451773188', 'request_id': 'req-64a2d10c-ba59-4699-957d-7161e20f4d76'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:57.021856 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-64a2d10c-ba59-4699-957d-7161e20f4d76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a47832d6f45f4805b784d19451773188', 'request_id': 'req-64a2d10c-ba59-4699-957d-7161e20f4d76'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:57.035124 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-64a2d10c-ba59-4699-957d-7161e20f4d76 admin admin] The token cache is being invalidate because project a47832d6f45f4805b784d19451773188 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:02:57.035446 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-64a2d10c-ba59-4699-957d-7161e20f4d76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:57.036142 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-64a2d10c-ba59-4699-957d-7161e20f4d76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:57.037767 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1418/2836] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 15:02:56 2022] DELETE /identity/v3/projects/a47832d6f45f4805b784d19451773188 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:02:57.041880 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934e640b-d1a9-4b10-8b70-5295779e609c service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:57.053367 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934e640b-d1a9-4b10-8b70-5295779e609c service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:57.110921 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bda9ee-5d64-407f-884d-fa67d3dd5709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:57.111830 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bda9ee-5d64-407f-884d-fa67d3dd5709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:57.112112 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bda9ee-5d64-407f-884d-fa67d3dd5709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:57.112386 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bda9ee-5d64-407f-884d-fa67d3dd5709 admin admin] PATH_INFO: `/v3/users/4e19f8285b574f17b63a09660fb50785` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:57.113687 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bda9ee-5d64-407f-884d-fa67d3dd5709 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e19f8285b574f17b63a09660fb50785)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:57.115662 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bda9ee-5d64-407f-884d-fa67d3dd5709 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:57.117158 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:57.168088 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-81bda9ee-5d64-407f-884d-fa67d3dd5709 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e19f8285b574f17b63a09660fb50785'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:57.169368 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-81bda9ee-5d64-407f-884d-fa67d3dd5709 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e19f8285b574f17b63a09660fb50785'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:57.181378 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-81bda9ee-5d64-407f-884d-fa67d3dd5709 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e19f8285b574f17b63a09660fb50785'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:57.182006 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-81bda9ee-5d64-407f-884d-fa67d3dd5709 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e19f8285b574f17b63a09660fb50785'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:57.189383 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-81bda9ee-5d64-407f-884d-fa67d3dd5709 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e19f8285b574f17b63a09660fb50785'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:57.201477 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1419/2837] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 15:02:57 2022] DELETE /identity/v3/users/4e19f8285b574f17b63a09660fb50785 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:57.472135 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2d10c-ba59-4699-957d-7161e20f4d76 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:57.473769 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-64a2d10c-ba59-4699-957d-7161e20f4d76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:57.487422 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2d10c-ba59-4699-957d-7161e20f4d76 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:57.553332 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5913bc-c33b-41a1-b47c-5477bf633f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:57.554246 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5913bc-c33b-41a1-b47c-5477bf633f13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:57.554533 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5913bc-c33b-41a1-b47c-5477bf633f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:57.554865 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5913bc-c33b-41a1-b47c-5477bf633f13 admin admin] PATH_INFO: `/v3/projects/76b82c295eb04443b2bc0595901ab41c` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:57.556114 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5913bc-c33b-41a1-b47c-5477bf633f13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76b82c295eb04443b2bc0595901ab41c)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:57.558096 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5913bc-c33b-41a1-b47c-5477bf633f13 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:57.608172 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0f5913bc-c33b-41a1-b47c-5477bf633f13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76b82c295eb04443b2bc0595901ab41c', 'request_id': 'req-0f5913bc-c33b-41a1-b47c-5477bf633f13'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:57.616217 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0f5913bc-c33b-41a1-b47c-5477bf633f13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76b82c295eb04443b2bc0595901ab41c', 'request_id': 'req-0f5913bc-c33b-41a1-b47c-5477bf633f13'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:57.623578 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0f5913bc-c33b-41a1-b47c-5477bf633f13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76b82c295eb04443b2bc0595901ab41c', 'request_id': 'req-0f5913bc-c33b-41a1-b47c-5477bf633f13'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:57.634172 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0f5913bc-c33b-41a1-b47c-5477bf633f13 admin admin] The token cache is being invalidate because project 76b82c295eb04443b2bc0595901ab41c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:02:57.634464 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0f5913bc-c33b-41a1-b47c-5477bf633f13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:57.635158 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0f5913bc-c33b-41a1-b47c-5477bf633f13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:57.639058 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1419/2838] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 15:02:57 2022] DELETE /identity/v3/projects/76b82c295eb04443b2bc0595901ab41c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:57.642111 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458190f4-ecb1-40a3-bbec-a7af7f85fdca None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:57.642615 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458190f4-ecb1-40a3-bbec-a7af7f85fdca None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:57.642711 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458190f4-ecb1-40a3-bbec-a7af7f85fdca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:57.663555 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-458190f4-ecb1-40a3-bbec-a7af7f85fdca None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:57.665457 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-458190f4-ecb1-40a3-bbec-a7af7f85fdca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:57.738422 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1420/2839] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 15:02:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:02:57.746092 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37521cd-a507-4cce-910b-908fd4da2929 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:57.746460 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37521cd-a507-4cce-910b-908fd4da2929 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:57.747435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37521cd-a507-4cce-910b-908fd4da2929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:57.766137 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-f37521cd-a507-4cce-910b-908fd4da2929 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:57.767485 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f37521cd-a507-4cce-910b-908fd4da2929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:57.775455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1420/2840] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 15:02:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:57.780252 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458190f4-ecb1-40a3-bbec-a7af7f85fdca None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:57.790137 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458190f4-ecb1-40a3-bbec-a7af7f85fdca None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:57.795434 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f82ca9-fdf7-4395-8aea-6e83845327fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:57.796346 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f82ca9-fdf7-4395-8aea-6e83845327fd admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:57.796643 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f82ca9-fdf7-4395-8aea-6e83845327fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:57.796924 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f82ca9-fdf7-4395-8aea-6e83845327fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:57.797668 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f82ca9-fdf7-4395-8aea-6e83845327fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:02:57.797984 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f82ca9-fdf7-4395-8aea-6e83845327fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:57.799631 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f82ca9-fdf7-4395-8aea-6e83845327fd admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:57.814435 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1421/2841] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 15:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:02:57.815727 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37521cd-a507-4cce-910b-908fd4da2929 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:57.824847 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37521cd-a507-4cce-910b-908fd4da2929 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:57.829517 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe811e-2ffe-44c5-82b9-1196354b1144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:57.830163 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe811e-2ffe-44c5-82b9-1196354b1144 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:57.830445 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe811e-2ffe-44c5-82b9-1196354b1144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:57.830722 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe811e-2ffe-44c5-82b9-1196354b1144 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:57.831448 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbe811e-2ffe-44c5-82b9-1196354b1144 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:57.833350 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbe811e-2ffe-44c5-82b9-1196354b1144 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:57.863037 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1421/2842] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 15:02:57 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:57.864301 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f82ca9-fdf7-4395-8aea-6e83845327fd admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:57.875137 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f82ca9-fdf7-4395-8aea-6e83845327fd admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:57.931085 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af264b4-f116-4d9f-89ba-8dda114b3dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:57.931760 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af264b4-f116-4d9f-89ba-8dda114b3dcd admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:57.932047 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af264b4-f116-4d9f-89ba-8dda114b3dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:57.932326 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af264b4-f116-4d9f-89ba-8dda114b3dcd admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:57.933014 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af264b4-f116-4d9f-89ba-8dda114b3dcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:57.934928 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af264b4-f116-4d9f-89ba-8dda114b3dcd admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:57.945396 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:57.956847 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1422/2843] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 15:02:57 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:57.960922 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe811e-2ffe-44c5-82b9-1196354b1144 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:57.971968 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe811e-2ffe-44c5-82b9-1196354b1144 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:57.976694 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20683341-7641-4f6d-a802-af0edeae51ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:57.977254 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20683341-7641-4f6d-a802-af0edeae51ab admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:57.977538 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20683341-7641-4f6d-a802-af0edeae51ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:57.977815 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20683341-7641-4f6d-a802-af0edeae51ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:57.978536 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20683341-7641-4f6d-a802-af0edeae51ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:57.980413 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20683341-7641-4f6d-a802-af0edeae51ab admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:57.993517 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1422/2844] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:57.998377 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af264b4-f116-4d9f-89ba-8dda114b3dcd admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:58.009567 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af264b4-f116-4d9f-89ba-8dda114b3dcd admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:58.014803 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb5402-f358-4bcf-a8b5-5ffef6db62ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:58.015491 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb5402-f358-4bcf-a8b5-5ffef6db62ac admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:58.015771 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb5402-f358-4bcf-a8b5-5ffef6db62ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:58.016044 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb5402-f358-4bcf-a8b5-5ffef6db62ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:58.017053 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceb5402-f358-4bcf-a8b5-5ffef6db62ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:58.018530 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceb5402-f358-4bcf-a8b5-5ffef6db62ac admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:58.033044 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1423/2845] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:58.037289 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20683341-7641-4f6d-a802-af0edeae51ab admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:58.047446 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20683341-7641-4f6d-a802-af0edeae51ab admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:58.052377 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c1beaf-760d-43c0-a7d0-dc53344ad764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:58.053245 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1beaf-760d-43c0-a7d0-dc53344ad764 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:58.053525 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1beaf-760d-43c0-a7d0-dc53344ad764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:58.053823 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1beaf-760d-43c0-a7d0-dc53344ad764 admin admin] PATH_INFO: `/v3/projects/0aa0a7f07c8c44f3b81d51322d705a23/users/734c2f52926840f4a89d882cd4f0fd78/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:58.068155 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c1beaf-760d-43c0-a7d0-dc53344ad764 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0aa0a7f07c8c44f3b81d51322d705a23, user_id=734c2f52926840f4a89d882cd4f0fd78, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:58.070818 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c1beaf-760d-43c0-a7d0-dc53344ad764 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:58.073816 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:58.086627 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1423/2846] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 15:02:58 2022] PUT /identity/v3/projects/0aa0a7f07c8c44f3b81d51322d705a23/users/734c2f52926840f4a89d882cd4f0fd78/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:58.097149 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6578f001-b738-4727-a7ae-fd447c0c5e10 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:58.097436 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6578f001-b738-4727-a7ae-fd447c0c5e10 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:58.097709 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6578f001-b738-4727-a7ae-fd447c0c5e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:58.123325 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-6578f001-b738-4727-a7ae-fd447c0c5e10 None None] MFA Rules not processed for user `734c2f52926840f4a89d882cd4f0fd78`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:58.124653 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6578f001-b738-4727-a7ae-fd447c0c5e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:58.189080 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1424/2847] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 15:02:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:58.193500 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c1beaf-760d-43c0-a7d0-dc53344ad764 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:58.261196 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c1beaf-760d-43c0-a7d0-dc53344ad764 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:58.266442 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322634c-2052-4683-9998-1d25ef165071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:58.267125 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322634c-2052-4683-9998-1d25ef165071 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:58.267410 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322634c-2052-4683-9998-1d25ef165071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:58.267683 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322634c-2052-4683-9998-1d25ef165071 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:58.268523 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9322634c-2052-4683-9998-1d25ef165071 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:58.270420 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9322634c-2052-4683-9998-1d25ef165071 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:58.298178 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1424/2848] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 15:02:58 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:58.302287 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6578f001-b738-4727-a7ae-fd447c0c5e10 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:58.313099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6578f001-b738-4727-a7ae-fd447c0c5e10 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:58.368816 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea9414-bce0-425f-8784-184e6eee1ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:58.369491 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea9414-bce0-425f-8784-184e6eee1ff7 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:58.369771 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea9414-bce0-425f-8784-184e6eee1ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:58.370043 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea9414-bce0-425f-8784-184e6eee1ff7 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:58.370736 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ea9414-bce0-425f-8784-184e6eee1ff7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:58.373248 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ea9414-bce0-425f-8784-184e6eee1ff7 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:58.396485 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1425/2849] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 15:02:58 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:58.397758 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322634c-2052-4683-9998-1d25ef165071 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:58.408103 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322634c-2052-4683-9998-1d25ef165071 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:58.412666 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925234c-1119-4582-b5f1-7afd7531f5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:58.413318 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925234c-1119-4582-b5f1-7afd7531f5d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:58.413604 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925234c-1119-4582-b5f1-7afd7531f5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:58.413881 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925234c-1119-4582-b5f1-7afd7531f5d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:58.414619 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7925234c-1119-4582-b5f1-7afd7531f5d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:58.416247 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7925234c-1119-4582-b5f1-7afd7531f5d4 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:58.427795 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1425/2850] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:58.431566 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea9414-bce0-425f-8784-184e6eee1ff7 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:58.442808 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea9414-bce0-425f-8784-184e6eee1ff7 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:58.447693 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2be5d2-54a5-445e-b70c-a352b8fb5350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:58.448440 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2be5d2-54a5-445e-b70c-a352b8fb5350 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:58.448730 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2be5d2-54a5-445e-b70c-a352b8fb5350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:58.449009 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2be5d2-54a5-445e-b70c-a352b8fb5350 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:58.449739 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2be5d2-54a5-445e-b70c-a352b8fb5350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:58.452024 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2be5d2-54a5-445e-b70c-a352b8fb5350 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:58.462581 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1426/2851] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:58.466812 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925234c-1119-4582-b5f1-7afd7531f5d4 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:58.477045 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925234c-1119-4582-b5f1-7afd7531f5d4 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:58.481926 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd9980-023d-48ff-8439-6d0e0fee5e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:58.482982 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd9980-023d-48ff-8439-6d0e0fee5e48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:58.483425 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd9980-023d-48ff-8439-6d0e0fee5e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:58.483773 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd9980-023d-48ff-8439-6d0e0fee5e48 admin admin] PATH_INFO: `/v3/projects/e70b0263878c4e089780d57571817dc1/users/ba385c562c7c48da85a91d119f9c46b6/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:58.501986 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd9980-023d-48ff-8439-6d0e0fee5e48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e70b0263878c4e089780d57571817dc1, user_id=ba385c562c7c48da85a91d119f9c46b6, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:58.504602 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd9980-023d-48ff-8439-6d0e0fee5e48 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:58.520080 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1426/2852] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 15:02:58 2022] PUT /identity/v3/projects/e70b0263878c4e089780d57571817dc1/users/ba385c562c7c48da85a91d119f9c46b6/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:58.585516 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c2a4b-1e2f-428e-964d-2db4ae2296ce None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:58.585833 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c2a4b-1e2f-428e-964d-2db4ae2296ce None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:58.586154 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c2a4b-1e2f-428e-964d-2db4ae2296ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:58.616822 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-334c2a4b-1e2f-428e-964d-2db4ae2296ce None None] MFA Rules not processed for user `ba385c562c7c48da85a91d119f9c46b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:58.618885 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-334c2a4b-1e2f-428e-964d-2db4ae2296ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:58.666363 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:58.691529 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1427/2853] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 15:02:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:58.695884 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd9980-023d-48ff-8439-6d0e0fee5e48 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:58.775867 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd9980-023d-48ff-8439-6d0e0fee5e48 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:58.780435 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287622c2-f835-49ba-b070-e4b5b06276a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:58.781095 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287622c2-f835-49ba-b070-e4b5b06276a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:58.781378 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287622c2-f835-49ba-b070-e4b5b06276a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:58.781652 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287622c2-f835-49ba-b070-e4b5b06276a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:58.782346 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287622c2-f835-49ba-b070-e4b5b06276a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:58.784430 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287622c2-f835-49ba-b070-e4b5b06276a2 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:58.823849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1427/2854] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 15:02:58 2022] POST /identity/v3/projects => generated 375 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:58.824976 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c2a4b-1e2f-428e-964d-2db4ae2296ce None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:58.835557 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c2a4b-1e2f-428e-964d-2db4ae2296ce None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:58.889913 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b183c-fb89-4855-b227-122463323a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:58.890588 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b183c-fb89-4855-b227-122463323a2a admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:58.890873 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b183c-fb89-4855-b227-122463323a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:58.891179 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b183c-fb89-4855-b227-122463323a2a admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:58.891869 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29b183c-fb89-4855-b227-122463323a2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:58.893887 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29b183c-fb89-4855-b227-122463323a2a admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:58.917079 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1428/2855] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 15:02:58 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:58.921846 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287622c2-f835-49ba-b070-e4b5b06276a2 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:58.937590 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287622c2-f835-49ba-b070-e4b5b06276a2 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:58.942812 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049e017-4d30-4712-9248-2635d591cc52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:58.943544 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049e017-4d30-4712-9248-2635d591cc52 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:58.943833 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049e017-4d30-4712-9248-2635d591cc52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:58.944114 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049e017-4d30-4712-9248-2635d591cc52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:58.944851 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6049e017-4d30-4712-9248-2635d591cc52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:58.946476 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6049e017-4d30-4712-9248-2635d591cc52 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:58.961491 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1428/2856] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:58.962675 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b183c-fb89-4855-b227-122463323a2a admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:58.973465 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b183c-fb89-4855-b227-122463323a2a admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:58.978369 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b58d0-398d-49e3-b499-6f2212c6cbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:58.979328 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b58d0-398d-49e3-b499-6f2212c6cbd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:58.979682 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b58d0-398d-49e3-b499-6f2212c6cbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:58.979953 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b58d0-398d-49e3-b499-6f2212c6cbd6 admin admin] PATH_INFO: `/v3/projects/e3d310059b9a4bf0bf7e74dd42d083a9/users/09b66ececa164e399642e57506cdca55/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:59.006142 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b58d0-398d-49e3-b499-6f2212c6cbd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3d310059b9a4bf0bf7e74dd42d083a9, user_id=09b66ececa164e399642e57506cdca55, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:59.009639 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b58d0-398d-49e3-b499-6f2212c6cbd6 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:59.028921 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1429/2857] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 15:02:58 2022] PUT /identity/v3/projects/e3d310059b9a4bf0bf7e74dd42d083a9/users/09b66ececa164e399642e57506cdca55/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:59.032576 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049e017-4d30-4712-9248-2635d591cc52 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:59.049582 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049e017-4d30-4712-9248-2635d591cc52 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:59.080104 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:59.121433 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ca767-12ff-42cb-b9b6-ce812cfb7fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:59.122633 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ca767-12ff-42cb-b9b6-ce812cfb7fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:59.122911 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ca767-12ff-42cb-b9b6-ce812cfb7fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:59.123211 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ca767-12ff-42cb-b9b6-ce812cfb7fb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:59.123926 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5ca767-12ff-42cb-b9b6-ce812cfb7fb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:59.125492 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5ca767-12ff-42cb-b9b6-ce812cfb7fb9 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:59.134920 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1429/2858] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:59.139728 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b58d0-398d-49e3-b499-6f2212c6cbd6 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:59.150042 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b58d0-398d-49e3-b499-6f2212c6cbd6 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:59.154664 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e41adc-4e43-47d8-b044-c85ff7cbaa3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:59.155590 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e41adc-4e43-47d8-b044-c85ff7cbaa3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:59.155878 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e41adc-4e43-47d8-b044-c85ff7cbaa3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:59.156156 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e41adc-4e43-47d8-b044-c85ff7cbaa3c admin admin] PATH_INFO: `/v3/projects/e3d310059b9a4bf0bf7e74dd42d083a9/users/09b66ececa164e399642e57506cdca55/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:59.158191 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e41adc-4e43-47d8-b044-c85ff7cbaa3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3d310059b9a4bf0bf7e74dd42d083a9, user_id=09b66ececa164e399642e57506cdca55, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:59.161036 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e41adc-4e43-47d8-b044-c85ff7cbaa3c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:59.171740 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1430/2859] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 15:02:59 2022] PUT /identity/v3/projects/e3d310059b9a4bf0bf7e74dd42d083a9/users/09b66ececa164e399642e57506cdca55/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:59.181349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22950279-5a3f-4cfc-93d6-f487c2f97a7b None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:59.181643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22950279-5a3f-4cfc-93d6-f487c2f97a7b None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:59.181928 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22950279-5a3f-4cfc-93d6-f487c2f97a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:59.208943 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-22950279-5a3f-4cfc-93d6-f487c2f97a7b None None] MFA Rules not processed for user `09b66ececa164e399642e57506cdca55`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:59.210196 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-22950279-5a3f-4cfc-93d6-f487c2f97a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:59.278224 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1430/2860] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 15:02:59 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:59.289342 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e41adc-4e43-47d8-b044-c85ff7cbaa3c admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:59.291064 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c8e41adc-4e43-47d8-b044-c85ff7cbaa3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:59.355815 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e41adc-4e43-47d8-b044-c85ff7cbaa3c admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:59.359736 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea1855-a9d5-48be-8e22-5771e0b921a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:59.360391 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea1855-a9d5-48be-8e22-5771e0b921a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:59.360679 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea1855-a9d5-48be-8e22-5771e0b921a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:59.360969 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea1855-a9d5-48be-8e22-5771e0b921a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:59.412921 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:59.426345 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea1855-a9d5-48be-8e22-5771e0b921a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:59.427888 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea1855-a9d5-48be-8e22-5771e0b921a8 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:59.438408 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1431/2861] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 15:02:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:03:00.942075 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22950279-5a3f-4cfc-93d6-f487c2f97a7b None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:00.943530 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-22950279-5a3f-4cfc-93d6-f487c2f97a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:00.955909 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22950279-5a3f-4cfc-93d6-f487c2f97a7b None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:00.959403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7824f4-d26f-40d4-a17d-5092430dbfe5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:00.960043 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7824f4-d26f-40d4-a17d-5092430dbfe5 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:00.960315 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7824f4-d26f-40d4-a17d-5092430dbfe5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:00.960585 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7824f4-d26f-40d4-a17d-5092430dbfe5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:01.018284 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7824f4-d26f-40d4-a17d-5092430dbfe5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:01.019782 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7824f4-d26f-40d4-a17d-5092430dbfe5 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:01.028879 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1431/2862] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 15:03:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:03:01.540475 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea1855-a9d5-48be-8e22-5771e0b921a8 service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:01.550216 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea1855-a9d5-48be-8e22-5771e0b921a8 service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:01.554015 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d31b78a-b040-4aca-ac6b-7301396d537f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b47cec76cb1c4afb8e011ff04a885f96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:01.554735 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31b78a-b040-4aca-ac6b-7301396d537f service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:01.555127 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31b78a-b040-4aca-ac6b-7301396d537f service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:01.555475 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31b78a-b040-4aca-ac6b-7301396d537f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:01.564855 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d31b78a-b040-4aca-ac6b-7301396d537f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:01.566288 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d31b78a-b040-4aca-ac6b-7301396d537f service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:01.576236 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1432/2863] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 15:03:01 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:03:01.718349 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7824f4-d26f-40d4-a17d-5092430dbfe5 service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:01.719772 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-4b7824f4-d26f-40d4-a17d-5092430dbfe5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:01.763791 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:01.777526 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7824f4-d26f-40d4-a17d-5092430dbfe5 service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:01.781659 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8de8d-4dc9-48f5-90fd-181613aa3a76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:01.782315 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8de8d-4dc9-48f5-90fd-181613aa3a76 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:01.782596 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8de8d-4dc9-48f5-90fd-181613aa3a76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:01.782871 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8de8d-4dc9-48f5-90fd-181613aa3a76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:01.784660 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-66d8de8d-4dc9-48f5-90fd-181613aa3a76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:01.849385 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d8de8d-4dc9-48f5-90fd-181613aa3a76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:01.850832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d8de8d-4dc9-48f5-90fd-181613aa3a76 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:01.860695 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1432/2864] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 15:03:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:03:05.567821 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d31b78a-b040-4aca-ac6b-7301396d537f service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:05.578925 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d31b78a-b040-4aca-ac6b-7301396d537f service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:05.583547 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca6cd1-6667-4870-9080-68397849f259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:05.584476 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca6cd1-6667-4870-9080-68397849f259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:05.584755 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca6cd1-6667-4870-9080-68397849f259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:05.585027 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca6cd1-6667-4870-9080-68397849f259 admin admin] PATH_INFO: `/v3/users/734c2f52926840f4a89d882cd4f0fd78` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:05.586504 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca6cd1-6667-4870-9080-68397849f259 admin admin] RBAC: Authorizing `identity:delete_user(user_id=734c2f52926840f4a89d882cd4f0fd78)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:05.588432 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca6cd1-6667-4870-9080-68397849f259 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:05.639133 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a4ca6cd1-6667-4870-9080-68397849f259 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '734c2f52926840f4a89d882cd4f0fd78'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:05.640022 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a4ca6cd1-6667-4870-9080-68397849f259 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '734c2f52926840f4a89d882cd4f0fd78'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:05.651890 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a4ca6cd1-6667-4870-9080-68397849f259 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '734c2f52926840f4a89d882cd4f0fd78'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:05.652540 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a4ca6cd1-6667-4870-9080-68397849f259 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '734c2f52926840f4a89d882cd4f0fd78'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:05.661458 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a4ca6cd1-6667-4870-9080-68397849f259 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '734c2f52926840f4a89d882cd4f0fd78'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:05.678939 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1433/2865] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 15:03:05 2022] DELETE /identity/v3/users/734c2f52926840f4a89d882cd4f0fd78 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:03:05.685902 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc727b-42a8-4bdb-820d-d19a1242f95d None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:05.686428 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc727b-42a8-4bdb-820d-d19a1242f95d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:05.686558 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc727b-42a8-4bdb-820d-d19a1242f95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:05.707010 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-c3dc727b-42a8-4bdb-820d-d19a1242f95d None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:05.708241 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c3dc727b-42a8-4bdb-820d-d19a1242f95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:05.769480 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1433/2866] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 15:03:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:03:05.779508 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca6cd1-6667-4870-9080-68397849f259 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:05.780948 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a4ca6cd1-6667-4870-9080-68397849f259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:05.842920 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca6cd1-6667-4870-9080-68397849f259 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:05.846829 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0063226-5aeb-4be8-86f0-cfc54ecd3c83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:05.847526 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0063226-5aeb-4be8-86f0-cfc54ecd3c83 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:05.848014 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0063226-5aeb-4be8-86f0-cfc54ecd3c83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:05.848125 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0063226-5aeb-4be8-86f0-cfc54ecd3c83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:05.859407 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0063226-5aeb-4be8-86f0-cfc54ecd3c83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:05.860864 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0063226-5aeb-4be8-86f0-cfc54ecd3c83 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:05.872845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1434/2867] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 15:03:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:03:06.336632 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc727b-42a8-4bdb-820d-d19a1242f95d None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:06.338443 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c3dc727b-42a8-4bdb-820d-d19a1242f95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:06.359062 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc727b-42a8-4bdb-820d-d19a1242f95d None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:06.365634 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d4375-0184-4ce1-a851-51653c4ffce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:06.367128 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d4375-0184-4ce1-a851-51653c4ffce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:06.367647 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d4375-0184-4ce1-a851-51653c4ffce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:06.368931 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d4375-0184-4ce1-a851-51653c4ffce6 admin admin] PATH_INFO: `/v3/projects/0aa0a7f07c8c44f3b81d51322d705a23` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:06.370612 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184d4375-0184-4ce1-a851-51653c4ffce6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0aa0a7f07c8c44f3b81d51322d705a23)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:06.372645 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184d4375-0184-4ce1-a851-51653c4ffce6 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:06.431124 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-184d4375-0184-4ce1-a851-51653c4ffce6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0aa0a7f07c8c44f3b81d51322d705a23', 'request_id': 'req-184d4375-0184-4ce1-a851-51653c4ffce6'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:06.440140 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-184d4375-0184-4ce1-a851-51653c4ffce6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0aa0a7f07c8c44f3b81d51322d705a23', 'request_id': 'req-184d4375-0184-4ce1-a851-51653c4ffce6'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:06.448870 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-184d4375-0184-4ce1-a851-51653c4ffce6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0aa0a7f07c8c44f3b81d51322d705a23', 'request_id': 'req-184d4375-0184-4ce1-a851-51653c4ffce6'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:06.462651 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-184d4375-0184-4ce1-a851-51653c4ffce6 admin admin] The token cache is being invalidate because project 0aa0a7f07c8c44f3b81d51322d705a23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:03:06.462949 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-184d4375-0184-4ce1-a851-51653c4ffce6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:06.463653 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-184d4375-0184-4ce1-a851-51653c4ffce6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:06.469394 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1434/2868] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 15:03:06 2022] DELETE /identity/v3/projects/0aa0a7f07c8c44f3b81d51322d705a23 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:03:06.473047 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0063226-5aeb-4be8-86f0-cfc54ecd3c83 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:06.484213 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0063226-5aeb-4be8-86f0-cfc54ecd3c83 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:06.537518 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:06.554130 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451bf016-5bc1-466e-8e5b-31c010e0dfc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:06.555043 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bf016-5bc1-466e-8e5b-31c010e0dfc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:06.555349 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bf016-5bc1-466e-8e5b-31c010e0dfc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:06.555611 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bf016-5bc1-466e-8e5b-31c010e0dfc5 admin admin] PATH_INFO: `/v3/users/ba385c562c7c48da85a91d119f9c46b6` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:06.557155 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451bf016-5bc1-466e-8e5b-31c010e0dfc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba385c562c7c48da85a91d119f9c46b6)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:06.559134 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451bf016-5bc1-466e-8e5b-31c010e0dfc5 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:06.611609 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-451bf016-5bc1-466e-8e5b-31c010e0dfc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba385c562c7c48da85a91d119f9c46b6'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:06.612991 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-451bf016-5bc1-466e-8e5b-31c010e0dfc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba385c562c7c48da85a91d119f9c46b6'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:06.626956 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-451bf016-5bc1-466e-8e5b-31c010e0dfc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba385c562c7c48da85a91d119f9c46b6'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:06.627667 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-451bf016-5bc1-466e-8e5b-31c010e0dfc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba385c562c7c48da85a91d119f9c46b6'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:06.638807 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-451bf016-5bc1-466e-8e5b-31c010e0dfc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba385c562c7c48da85a91d119f9c46b6'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:06.653519 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1435/2869] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 15:03:06 2022] DELETE /identity/v3/users/ba385c562c7c48da85a91d119f9c46b6 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:03:06.970279 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d4375-0184-4ce1-a851-51653c4ffce6 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:06.972168 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-184d4375-0184-4ce1-a851-51653c4ffce6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:06.987361 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d4375-0184-4ce1-a851-51653c4ffce6 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:06.999562 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:07.061861 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b95bbb-d239-455d-9c38-a9a1b2916ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:07.062547 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b95bbb-d239-455d-9c38-a9a1b2916ede admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:07.062899 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b95bbb-d239-455d-9c38-a9a1b2916ede admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:07.063270 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b95bbb-d239-455d-9c38-a9a1b2916ede admin admin] PATH_INFO: `/v3/projects/e70b0263878c4e089780d57571817dc1` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:07.064592 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b95bbb-d239-455d-9c38-a9a1b2916ede admin admin] RBAC: Authorizing `identity:delete_project(project_id=e70b0263878c4e089780d57571817dc1)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:07.067090 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b95bbb-d239-455d-9c38-a9a1b2916ede admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:07.116217 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-31b95bbb-d239-455d-9c38-a9a1b2916ede admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e70b0263878c4e089780d57571817dc1', 'request_id': 'req-31b95bbb-d239-455d-9c38-a9a1b2916ede'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:07.125340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-31b95bbb-d239-455d-9c38-a9a1b2916ede admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e70b0263878c4e089780d57571817dc1', 'request_id': 'req-31b95bbb-d239-455d-9c38-a9a1b2916ede'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:07.132044 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-31b95bbb-d239-455d-9c38-a9a1b2916ede admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e70b0263878c4e089780d57571817dc1', 'request_id': 'req-31b95bbb-d239-455d-9c38-a9a1b2916ede'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:07.142210 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-31b95bbb-d239-455d-9c38-a9a1b2916ede admin admin] The token cache is being invalidate because project e70b0263878c4e089780d57571817dc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:03:07.142497 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-31b95bbb-d239-455d-9c38-a9a1b2916ede admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:07.143168 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-31b95bbb-d239-455d-9c38-a9a1b2916ede admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:07.144713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1435/2870] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 15:03:06 2022] DELETE /identity/v3/projects/e70b0263878c4e089780d57571817dc1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:03:07.148934 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451bf016-5bc1-466e-8e5b-31c010e0dfc5 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:07.150448 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-451bf016-5bc1-466e-8e5b-31c010e0dfc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:07.163811 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451bf016-5bc1-466e-8e5b-31c010e0dfc5 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:07.223308 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c26cf9-c5b6-4ffb-9758-6b941fb9536d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:07.224218 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c26cf9-c5b6-4ffb-9758-6b941fb9536d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:07.224498 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c26cf9-c5b6-4ffb-9758-6b941fb9536d admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:07.224774 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c26cf9-c5b6-4ffb-9758-6b941fb9536d admin admin] PATH_INFO: `/v3/users/09b66ececa164e399642e57506cdca55` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:07.226001 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c26cf9-c5b6-4ffb-9758-6b941fb9536d admin admin] RBAC: Authorizing `identity:delete_user(user_id=09b66ececa164e399642e57506cdca55)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:07.228031 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c26cf9-c5b6-4ffb-9758-6b941fb9536d admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:07.282049 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-04c26cf9-c5b6-4ffb-9758-6b941fb9536d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09b66ececa164e399642e57506cdca55'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:07.282719 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-04c26cf9-c5b6-4ffb-9758-6b941fb9536d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09b66ececa164e399642e57506cdca55'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:07.295584 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-04c26cf9-c5b6-4ffb-9758-6b941fb9536d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09b66ececa164e399642e57506cdca55'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:07.297118 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-04c26cf9-c5b6-4ffb-9758-6b941fb9536d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09b66ececa164e399642e57506cdca55'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:07.304817 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-04c26cf9-c5b6-4ffb-9758-6b941fb9536d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09b66ececa164e399642e57506cdca55'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:07.306457 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:07.325745 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1436/2871] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 15:03:07 2022] DELETE /identity/v3/users/09b66ececa164e399642e57506cdca55 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:03:07.631811 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b95bbb-d239-455d-9c38-a9a1b2916ede admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:07.633403 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-31b95bbb-d239-455d-9c38-a9a1b2916ede admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:07.650454 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b95bbb-d239-455d-9c38-a9a1b2916ede admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:07.755755 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc661f51-9663-4060-888f-3b9cc8a1a33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:07.757037 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc661f51-9663-4060-888f-3b9cc8a1a33d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:07.757476 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc661f51-9663-4060-888f-3b9cc8a1a33d admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:07.757832 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc661f51-9663-4060-888f-3b9cc8a1a33d admin admin] PATH_INFO: `/v3/projects/e3d310059b9a4bf0bf7e74dd42d083a9` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:07.759753 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc661f51-9663-4060-888f-3b9cc8a1a33d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3d310059b9a4bf0bf7e74dd42d083a9)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:07.764782 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc661f51-9663-4060-888f-3b9cc8a1a33d admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:07.837673 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:07.843979 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-fc661f51-9663-4060-888f-3b9cc8a1a33d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3d310059b9a4bf0bf7e74dd42d083a9', 'request_id': 'req-fc661f51-9663-4060-888f-3b9cc8a1a33d'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:07.855796 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-fc661f51-9663-4060-888f-3b9cc8a1a33d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3d310059b9a4bf0bf7e74dd42d083a9', 'request_id': 'req-fc661f51-9663-4060-888f-3b9cc8a1a33d'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:07.865238 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-fc661f51-9663-4060-888f-3b9cc8a1a33d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3d310059b9a4bf0bf7e74dd42d083a9', 'request_id': 'req-fc661f51-9663-4060-888f-3b9cc8a1a33d'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:07.881466 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-fc661f51-9663-4060-888f-3b9cc8a1a33d admin admin] The token cache is being invalidate because project e3d310059b9a4bf0bf7e74dd42d083a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:03:07.881806 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-fc661f51-9663-4060-888f-3b9cc8a1a33d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:07.882612 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-fc661f51-9663-4060-888f-3b9cc8a1a33d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:07.885020 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1436/2872] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 15:03:07 2022] DELETE /identity/v3/projects/e3d310059b9a4bf0bf7e74dd42d083a9 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:03:07.893372 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab285da-4067-47b8-8f2a-4fc38306e6a4 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:07.893929 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab285da-4067-47b8-8f2a-4fc38306e6a4 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:07.894752 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab285da-4067-47b8-8f2a-4fc38306e6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:07.932868 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-3ab285da-4067-47b8-8f2a-4fc38306e6a4 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:07.935041 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-3ab285da-4067-47b8-8f2a-4fc38306e6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:08.030750 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a2dfc-927d-4937-b849-4cdf202b8a38 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:08.031363 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a2dfc-927d-4937-b849-4cdf202b8a38 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:08.031570 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a2dfc-927d-4937-b849-4cdf202b8a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:08.039675 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1437/2873] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 15:03:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:03:08.054198 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-f31a2dfc-927d-4937-b849-4cdf202b8a38 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:08.054927 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f31a2dfc-927d-4937-b849-4cdf202b8a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:08.066183 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1437/2874] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 15:03:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:03:08.070876 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab285da-4067-47b8-8f2a-4fc38306e6a4 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:08.082987 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab285da-4067-47b8-8f2a-4fc38306e6a4 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:08.089291 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c221e815-a71d-4346-9c6d-5ef74fffa45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:08.089971 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221e815-a71d-4346-9c6d-5ef74fffa45a admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:08.091417 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221e815-a71d-4346-9c6d-5ef74fffa45a admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:08.091417 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221e815-a71d-4346-9c6d-5ef74fffa45a admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:08.091912 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c221e815-a71d-4346-9c6d-5ef74fffa45a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:03:08.092222 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c221e815-a71d-4346-9c6d-5ef74fffa45a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:08.094117 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c221e815-a71d-4346-9c6d-5ef74fffa45a admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:08.109086 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1438/2875] 10.222.0.45 () {58 vars in 1152 bytes} [Wed Jun 15 15:03:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:03:08.113248 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a2dfc-927d-4937-b849-4cdf202b8a38 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:08.125296 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a2dfc-927d-4937-b849-4cdf202b8a38 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:08.134507 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee26a81-bfc7-40ef-8368-11176009b3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:08.135217 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee26a81-bfc7-40ef-8368-11176009b3bb admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:08.135499 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee26a81-bfc7-40ef-8368-11176009b3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:08.135773 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee26a81-bfc7-40ef-8368-11176009b3bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:08.136490 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee26a81-bfc7-40ef-8368-11176009b3bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:08.139208 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee26a81-bfc7-40ef-8368-11176009b3bb admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:08.174434 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1438/2876] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 15:03:08 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:03:08.178499 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c221e815-a71d-4346-9c6d-5ef74fffa45a admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:08.194977 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c221e815-a71d-4346-9c6d-5ef74fffa45a admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:08.283274 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37f7c7-6013-454a-a1a5-0be8e286c137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:08.283971 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37f7c7-6013-454a-a1a5-0be8e286c137 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:08.284394 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37f7c7-6013-454a-a1a5-0be8e286c137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:08.284688 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37f7c7-6013-454a-a1a5-0be8e286c137 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:08.285410 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c37f7c7-6013-454a-a1a5-0be8e286c137 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:08.288385 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c37f7c7-6013-454a-a1a5-0be8e286c137 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:08.312687 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1439/2877] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 15:03:08 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:03:08.316658 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee26a81-bfc7-40ef-8368-11176009b3bb admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:08.328541 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee26a81-bfc7-40ef-8368-11176009b3bb admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:08.334935 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da2b3f-9a8a-4cf9-a8c3-b0ffddb26761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:08.336064 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da2b3f-9a8a-4cf9-a8c3-b0ffddb26761 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:08.336478 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da2b3f-9a8a-4cf9-a8c3-b0ffddb26761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:08.336849 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da2b3f-9a8a-4cf9-a8c3-b0ffddb26761 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:08.337682 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da2b3f-9a8a-4cf9-a8c3-b0ffddb26761 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:08.339621 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da2b3f-9a8a-4cf9-a8c3-b0ffddb26761 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:08.353959 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1439/2878] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 15:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:03:08.358344 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37f7c7-6013-454a-a1a5-0be8e286c137 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:08.370347 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37f7c7-6013-454a-a1a5-0be8e286c137 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:08.375723 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f628a-7c3c-4984-ba88-b200498f836b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:08.376507 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f628a-7c3c-4984-ba88-b200498f836b admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:08.376841 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f628a-7c3c-4984-ba88-b200498f836b admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:08.377183 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f628a-7c3c-4984-ba88-b200498f836b admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:08.377999 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f628a-7c3c-4984-ba88-b200498f836b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:08.379731 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f628a-7c3c-4984-ba88-b200498f836b admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:08.395260 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1440/2879] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 15:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:03:08.396648 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da2b3f-9a8a-4cf9-a8c3-b0ffddb26761 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:08.408325 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da2b3f-9a8a-4cf9-a8c3-b0ffddb26761 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:08.423946 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a3129-1a44-4a40-a2ae-a60cbd2558f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:08.424989 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a3129-1a44-4a40-a2ae-a60cbd2558f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:08.425368 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a3129-1a44-4a40-a2ae-a60cbd2558f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:08.425744 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a3129-1a44-4a40-a2ae-a60cbd2558f8 admin admin] PATH_INFO: `/v3/projects/c54a1f9aea1a42bfb5fc36770120eb58/users/abf3c553d53f4118a2b4bb6ba27debe5/roles/ce0de9b9c9054826b569db895942ad26` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:08.441946 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f628a-7c3c-4984-ba88-b200498f836b admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:08.442137 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a3129-1a44-4a40-a2ae-a60cbd2558f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c54a1f9aea1a42bfb5fc36770120eb58, user_id=abf3c553d53f4118a2b4bb6ba27debe5, role_id=ce0de9b9c9054826b569db895942ad26)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:08.444191 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a76f628a-7c3c-4984-ba88-b200498f836b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:08.444966 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a3129-1a44-4a40-a2ae-a60cbd2558f8 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:08.459731 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1440/2880] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 15:03:08 2022] PUT /identity/v3/projects/c54a1f9aea1a42bfb5fc36770120eb58/users/abf3c553d53f4118a2b4bb6ba27debe5/roles/ce0de9b9c9054826b569db895942ad26 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:03:08.464964 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c612334-7acb-464e-9451-af5691daf0d9 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:08.465493 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c612334-7acb-464e-9451-af5691daf0d9 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:08.466541 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c612334-7acb-464e-9451-af5691daf0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:08.485458 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-8c612334-7acb-464e-9451-af5691daf0d9 None None] MFA Rules not processed for user `a476d9b478cd40989e9e4e02fbc699e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:08.487246 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-8c612334-7acb-464e-9451-af5691daf0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:08.526870 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:08.544968 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f628a-7c3c-4984-ba88-b200498f836b admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:08.557214 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1441/2881] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 15:03:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:03:08.558082 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c612334-7acb-464e-9451-af5691daf0d9 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:08.559501 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-8c612334-7acb-464e-9451-af5691daf0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:08.571973 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c612334-7acb-464e-9451-af5691daf0d9 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:08.577287 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c2d91a-e614-4425-8031-ff29e1bda61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:08.578271 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c2d91a-e614-4425-8031-ff29e1bda61d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:08.578566 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c2d91a-e614-4425-8031-ff29e1bda61d admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:08.578851 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c2d91a-e614-4425-8031-ff29e1bda61d admin admin] PATH_INFO: `/v3/users/1b23fa0b7fdb4016917987ea1c40af33` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:08.580095 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c2d91a-e614-4425-8031-ff29e1bda61d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b23fa0b7fdb4016917987ea1c40af33)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:08.582011 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c2d91a-e614-4425-8031-ff29e1bda61d admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:08.610868 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1444ebb9-8e88-4451-97c7-a18afabce64f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4128111a9aa450abb2146a5d9926dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:08.612781 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444ebb9-8e88-4451-97c7-a18afabce64f service placement] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:08.612781 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444ebb9-8e88-4451-97c7-a18afabce64f service placement] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:08.612781 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444ebb9-8e88-4451-97c7-a18afabce64f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:08.614349 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1444ebb9-8e88-4451-97c7-a18afabce64f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:08.628269 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1444ebb9-8e88-4451-97c7-a18afabce64f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:08.629874 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1444ebb9-8e88-4451-97c7-a18afabce64f service placement] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:08.630809 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c1c2d91a-e614-4425-8031-ff29e1bda61d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b23fa0b7fdb4016917987ea1c40af33'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:08.631509 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c1c2d91a-e614-4425-8031-ff29e1bda61d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b23fa0b7fdb4016917987ea1c40af33'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:08.641361 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1441/2882] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 15:03:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:03:08.642536 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1444ebb9-8e88-4451-97c7-a18afabce64f service placement] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:08.643890 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1444ebb9-8e88-4451-97c7-a18afabce64f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:08.644945 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c1c2d91a-e614-4425-8031-ff29e1bda61d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b23fa0b7fdb4016917987ea1c40af33'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:08.646888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:08.655603 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c1c2d91a-e614-4425-8031-ff29e1bda61d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b23fa0b7fdb4016917987ea1c40af33'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:08.664879 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c1c2d91a-e614-4425-8031-ff29e1bda61d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b23fa0b7fdb4016917987ea1c40af33'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:08.671295 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1442/2883] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 15:03:08 2022] DELETE /identity/v3/users/1b23fa0b7fdb4016917987ea1c40af33 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:03:08.714871 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1444ebb9-8e88-4451-97c7-a18afabce64f service placement] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:08.766457 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706026a8-5450-4ac3-871b-dfdbbfd2be45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:08.767175 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706026a8-5450-4ac3-871b-dfdbbfd2be45 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:08.767471 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706026a8-5450-4ac3-871b-dfdbbfd2be45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:08.767738 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706026a8-5450-4ac3-871b-dfdbbfd2be45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:08.769614 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-706026a8-5450-4ac3-871b-dfdbbfd2be45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:08.830228 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:08.834564 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706026a8-5450-4ac3-871b-dfdbbfd2be45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:08.836680 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706026a8-5450-4ac3-871b-dfdbbfd2be45 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:08.850504 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1442/2884] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 15:03:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:03:08.851686 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c2d91a-e614-4425-8031-ff29e1bda61d admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:08.853097 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c1c2d91a-e614-4425-8031-ff29e1bda61d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:08.866416 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c2d91a-e614-4425-8031-ff29e1bda61d admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:08.872434 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5311034-2400-463f-b7ab-ab3ea3dc894a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:08.873321 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5311034-2400-463f-b7ab-ab3ea3dc894a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:08.873608 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5311034-2400-463f-b7ab-ab3ea3dc894a admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:08.873888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5311034-2400-463f-b7ab-ab3ea3dc894a admin admin] PATH_INFO: `/v3/projects/59120b6715c1485db0cb1277a52e6c55` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:08.875149 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5311034-2400-463f-b7ab-ab3ea3dc894a admin admin] RBAC: Authorizing `identity:delete_project(project_id=59120b6715c1485db0cb1277a52e6c55)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:08.877142 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5311034-2400-463f-b7ab-ab3ea3dc894a admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:08.931881 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b5311034-2400-463f-b7ab-ab3ea3dc894a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59120b6715c1485db0cb1277a52e6c55', 'request_id': 'req-b5311034-2400-463f-b7ab-ab3ea3dc894a'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:08.941058 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b5311034-2400-463f-b7ab-ab3ea3dc894a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59120b6715c1485db0cb1277a52e6c55', 'request_id': 'req-b5311034-2400-463f-b7ab-ab3ea3dc894a'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:08.947858 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b5311034-2400-463f-b7ab-ab3ea3dc894a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59120b6715c1485db0cb1277a52e6c55', 'request_id': 'req-b5311034-2400-463f-b7ab-ab3ea3dc894a'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:08.959652 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b5311034-2400-463f-b7ab-ab3ea3dc894a admin admin] The token cache is being invalidate because project 59120b6715c1485db0cb1277a52e6c55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:03:08.959944 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b5311034-2400-463f-b7ab-ab3ea3dc894a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:08.960621 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b5311034-2400-463f-b7ab-ab3ea3dc894a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:08.962211 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1443/2885] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 15:03:08 2022] DELETE /identity/v3/projects/59120b6715c1485db0cb1277a52e6c55 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:03:08.966313 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706026a8-5450-4ac3-871b-dfdbbfd2be45 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:08.976867 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706026a8-5450-4ac3-871b-dfdbbfd2be45 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:09.044909 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae622e-9ad9-41a2-9f1e-e7d84f038b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:09.045802 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae622e-9ad9-41a2-9f1e-e7d84f038b6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:09.046092 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae622e-9ad9-41a2-9f1e-e7d84f038b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:09.046652 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae622e-9ad9-41a2-9f1e-e7d84f038b6f admin admin] PATH_INFO: `/v3/users/ff97fc23806243179bd7b1d18d1ec811` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:09.047912 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae622e-9ad9-41a2-9f1e-e7d84f038b6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff97fc23806243179bd7b1d18d1ec811)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:09.049871 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae622e-9ad9-41a2-9f1e-e7d84f038b6f admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:09.100666 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-70ae622e-9ad9-41a2-9f1e-e7d84f038b6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff97fc23806243179bd7b1d18d1ec811'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:09.101457 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-70ae622e-9ad9-41a2-9f1e-e7d84f038b6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff97fc23806243179bd7b1d18d1ec811'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:09.114649 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-70ae622e-9ad9-41a2-9f1e-e7d84f038b6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff97fc23806243179bd7b1d18d1ec811'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:09.120893 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-70ae622e-9ad9-41a2-9f1e-e7d84f038b6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff97fc23806243179bd7b1d18d1ec811'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:09.131424 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-70ae622e-9ad9-41a2-9f1e-e7d84f038b6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff97fc23806243179bd7b1d18d1ec811'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:09.154797 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1443/2886] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 15:03:08 2022] DELETE /identity/v3/users/ff97fc23806243179bd7b1d18d1ec811 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:03:09.367187 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5311034-2400-463f-b7ab-ab3ea3dc894a admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:09.369386 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b5311034-2400-463f-b7ab-ab3ea3dc894a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:09.385313 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5311034-2400-463f-b7ab-ab3ea3dc894a admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:09.452037 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5baff-f46d-49ca-a4a1-600946eca511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:09.452825 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5baff-f46d-49ca-a4a1-600946eca511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:09.453408 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5baff-f46d-49ca-a4a1-600946eca511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:09.453408 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5baff-f46d-49ca-a4a1-600946eca511 admin admin] PATH_INFO: `/v3/projects/2ac479c9f83943b0801b60719412e9c4` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:09.454583 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e5baff-f46d-49ca-a4a1-600946eca511 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ac479c9f83943b0801b60719412e9c4)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:09.456583 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e5baff-f46d-49ca-a4a1-600946eca511 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:09.507318 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-16e5baff-f46d-49ca-a4a1-600946eca511 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ac479c9f83943b0801b60719412e9c4', 'request_id': 'req-16e5baff-f46d-49ca-a4a1-600946eca511'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:09.517643 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-16e5baff-f46d-49ca-a4a1-600946eca511 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ac479c9f83943b0801b60719412e9c4', 'request_id': 'req-16e5baff-f46d-49ca-a4a1-600946eca511'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:09.524897 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-16e5baff-f46d-49ca-a4a1-600946eca511 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ac479c9f83943b0801b60719412e9c4', 'request_id': 'req-16e5baff-f46d-49ca-a4a1-600946eca511'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:09.531455 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:INSERT=3,keystone:DELETE=15,keystone:SELECT=82 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:09.541426 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-16e5baff-f46d-49ca-a4a1-600946eca511 admin admin] The token cache is being invalidate because project 2ac479c9f83943b0801b60719412e9c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:03:09.541822 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-16e5baff-f46d-49ca-a4a1-600946eca511 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:09.542394 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-16e5baff-f46d-49ca-a4a1-600946eca511 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:09.545724 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1444/2887] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 15:03:09 2022] DELETE /identity/v3/projects/2ac479c9f83943b0801b60719412e9c4 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:03:12.758757 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017089e9-91ed-405f-8560-0791310879fe None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:12.759347 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017089e9-91ed-405f-8560-0791310879fe None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:12.759347 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017089e9-91ed-405f-8560-0791310879fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:12.787644 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-017089e9-91ed-405f-8560-0791310879fe None None] MFA Rules not processed for user `abf3c553d53f4118a2b4bb6ba27debe5`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:12.788805 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-017089e9-91ed-405f-8560-0791310879fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:12.859672 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1444/2888] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 15:03:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:03:12.864121 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5baff-f46d-49ca-a4a1-600946eca511 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:12.865543 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-16e5baff-f46d-49ca-a4a1-600946eca511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:12.961604 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5baff-f46d-49ca-a4a1-600946eca511 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:12.966718 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00949192-bb60-4b44-baf3-b979ad8ceefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:12.967868 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00949192-bb60-4b44-baf3-b979ad8ceefc admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:12.968373 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00949192-bb60-4b44-baf3-b979ad8ceefc admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:12.968878 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00949192-bb60-4b44-baf3-b979ad8ceefc admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:12.969615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00949192-bb60-4b44-baf3-b979ad8ceefc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:12.972129 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00949192-bb60-4b44-baf3-b979ad8ceefc admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:13.002559 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1445/2889] 10.222.0.45 () {60 vars in 1152 bytes} [Wed Jun 15 15:03:12 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:03:13.006577 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017089e9-91ed-405f-8560-0791310879fe None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:13.013263 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:13.019685 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017089e9-91ed-405f-8560-0791310879fe None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:13.087244 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3effdd7c-ef0d-411e-91d4-235770737653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:13.087888 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3effdd7c-ef0d-411e-91d4-235770737653 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:13.088277 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3effdd7c-ef0d-411e-91d4-235770737653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:13.088430 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3effdd7c-ef0d-411e-91d4-235770737653 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:13.089170 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3effdd7c-ef0d-411e-91d4-235770737653 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:13.091292 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3effdd7c-ef0d-411e-91d4-235770737653 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:13.116162 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1445/2890] 10.222.0.45 () {60 vars in 1143 bytes} [Wed Jun 15 15:03:13 2022] POST /identity/v3/users => generated 299 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:03:13.119770 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00949192-bb60-4b44-baf3-b979ad8ceefc admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:13.131143 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00949192-bb60-4b44-baf3-b979ad8ceefc admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:13.136561 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc3ac55-f144-4b6e-9d7b-de965fad77b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:13.137368 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3ac55-f144-4b6e-9d7b-de965fad77b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:13.137697 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3ac55-f144-4b6e-9d7b-de965fad77b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:13.137980 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3ac55-f144-4b6e-9d7b-de965fad77b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:13.138713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc3ac55-f144-4b6e-9d7b-de965fad77b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:13.140423 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc3ac55-f144-4b6e-9d7b-de965fad77b6 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:13.151762 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1446/2891] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 15:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:03:13.156284 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3effdd7c-ef0d-411e-91d4-235770737653 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:13.166435 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3effdd7c-ef0d-411e-91d4-235770737653 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:13.172007 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc25dc73-09f8-49eb-8114-b2b25e7c963e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:13.172975 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25dc73-09f8-49eb-8114-b2b25e7c963e admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:13.173259 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25dc73-09f8-49eb-8114-b2b25e7c963e admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:13.173535 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25dc73-09f8-49eb-8114-b2b25e7c963e admin admin] PATH_INFO: `/v3/projects/b4aec680871c4bb4b4f2a657050972dd/users/afd6be4d9dcb461fbed484c1271f0006/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:13.189840 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc25dc73-09f8-49eb-8114-b2b25e7c963e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4aec680871c4bb4b4f2a657050972dd, user_id=afd6be4d9dcb461fbed484c1271f0006, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:13.192530 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc25dc73-09f8-49eb-8114-b2b25e7c963e admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:13.206971 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1446/2892] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 15:03:13 2022] PUT /identity/v3/projects/b4aec680871c4bb4b4f2a657050972dd/users/afd6be4d9dcb461fbed484c1271f0006/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:03:13.210977 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc3ac55-f144-4b6e-9d7b-de965fad77b6 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:13.221650 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc3ac55-f144-4b6e-9d7b-de965fad77b6 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:13.285678 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de7b12-5750-43b4-a5ea-2d25d086b801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:13.286432 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de7b12-5750-43b4-a5ea-2d25d086b801 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:13.286713 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de7b12-5750-43b4-a5ea-2d25d086b801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:13.287058 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de7b12-5750-43b4-a5ea-2d25d086b801 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:13.287816 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de7b12-5750-43b4-a5ea-2d25d086b801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:13.289405 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de7b12-5750-43b4-a5ea-2d25d086b801 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:13.301035 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1447/2893] 10.222.0.45 () {58 vars in 1121 bytes} [Wed Jun 15 15:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:03:13.305267 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc25dc73-09f8-49eb-8114-b2b25e7c963e admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:13.332845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc25dc73-09f8-49eb-8114-b2b25e7c963e admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:13.337832 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d408b442-dcc7-4363-8390-dba004e06da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:13.338753 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d408b442-dcc7-4363-8390-dba004e06da3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:13.339063 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d408b442-dcc7-4363-8390-dba004e06da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:13.339348 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d408b442-dcc7-4363-8390-dba004e06da3 admin admin] PATH_INFO: `/v3/projects/b4aec680871c4bb4b4f2a657050972dd/users/afd6be4d9dcb461fbed484c1271f0006/roles/ed1cfa2fcf724a4d84f5acea100930c7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:13.341868 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d408b442-dcc7-4363-8390-dba004e06da3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4aec680871c4bb4b4f2a657050972dd, user_id=afd6be4d9dcb461fbed484c1271f0006, role_id=ed1cfa2fcf724a4d84f5acea100930c7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:13.344530 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d408b442-dcc7-4363-8390-dba004e06da3 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:13.362008 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1447/2894] 10.222.0.45 () {60 vars in 1482 bytes} [Wed Jun 15 15:03:13 2022] PUT /identity/v3/projects/b4aec680871c4bb4b4f2a657050972dd/users/afd6be4d9dcb461fbed484c1271f0006/roles/ed1cfa2fcf724a4d84f5acea100930c7 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:03:17.647543 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150bf38a-034e-4e8a-977c-2f06029e09e4 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:17.648002 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150bf38a-034e-4e8a-977c-2f06029e09e4 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:17.648121 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150bf38a-034e-4e8a-977c-2f06029e09e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:17.677690 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-150bf38a-034e-4e8a-977c-2f06029e09e4 None None] MFA Rules not processed for user `afd6be4d9dcb461fbed484c1271f0006`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:17.679015 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-150bf38a-034e-4e8a-977c-2f06029e09e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:17.738258 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:17.760680 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1448/2895] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 15:03:17 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:03:17.772910 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d408b442-dcc7-4363-8390-dba004e06da3 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:17.774616 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d408b442-dcc7-4363-8390-dba004e06da3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:17.853873 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d408b442-dcc7-4363-8390-dba004e06da3 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:17.858077 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f7ec2-88aa-4a5f-926f-7486c12c6943 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f10214b2bb4eafb3dff7adf71b9bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:17.858754 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f7ec2-88aa-4a5f-926f-7486c12c6943 service glance] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:17.859062 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f7ec2-88aa-4a5f-926f-7486c12c6943 service glance] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:17.859344 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f7ec2-88aa-4a5f-926f-7486c12c6943 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:17.909263 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:17.920936 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200f7ec2-88aa-4a5f-926f-7486c12c6943 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:17.922642 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200f7ec2-88aa-4a5f-926f-7486c12c6943 service glance] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:17.932038 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1448/2896] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 15:03:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:03:18.031600 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1592a-8fcc-4548-911f-9bffe2783163 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:18.031913 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1592a-8fcc-4548-911f-9bffe2783163 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:18.032192 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1592a-8fcc-4548-911f-9bffe2783163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:18.101760 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-fcf1592a-8fcc-4548-911f-9bffe2783163 None None] MFA Rules not processed for user `bf393488726e49338e70017f5cb7c4be`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:18.103013 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-fcf1592a-8fcc-4548-911f-9bffe2783163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:18.169478 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1449/2897] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Jun 15 15:03:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:03:18.221109 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f7ec2-88aa-4a5f-926f-7486c12c6943 service glance] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:18.222511 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-200f7ec2-88aa-4a5f-926f-7486c12c6943 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:18.332285 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f7ec2-88aa-4a5f-926f-7486c12c6943 service glance] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:18.339124 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f7316-ba1c-4f07-a4a0-d1c267242cde service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '641a81b3140f47119b5c2323a97a6681', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:18.339793 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f7316-ba1c-4f07-a4a0-d1c267242cde service swift] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:18.340171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f7316-ba1c-4f07-a4a0-d1c267242cde service swift] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:18.340386 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f7316-ba1c-4f07-a4a0-d1c267242cde service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:18.351806 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61f7316-ba1c-4f07-a4a0-d1c267242cde service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:18.353664 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61f7316-ba1c-4f07-a4a0-d1c267242cde service swift] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:18.363417 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1449/2898] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Jun 15 15:03:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 15:03:27.030745 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4707c-38e0-484f-a905-b02611f7dddd None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:27.031311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4707c-38e0-484f-a905-b02611f7dddd None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:27.031311 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4707c-38e0-484f-a905-b02611f7dddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:27.048837 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-e9c4707c-38e0-484f-a905-b02611f7dddd None None] MFA Rules not processed for user `abf3c553d53f4118a2b4bb6ba27debe5`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:27.050046 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e9c4707c-38e0-484f-a905-b02611f7dddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:27.056951 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1450/2899] 10.222.0.45 () {58 vars in 957 bytes} [Wed Jun 15 15:03:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:03:27.616533 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f7316-ba1c-4f07-a4a0-d1c267242cde service swift] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:27.618016 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a61f7316-ba1c-4f07-a4a0-d1c267242cde service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:27.678629 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f7316-ba1c-4f07-a4a0-d1c267242cde service swift] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:27.682545 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c23e34-ca22-46b9-9161-623da8448395 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:27.683228 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c23e34-ca22-46b9-9161-623da8448395 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:27.683512 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c23e34-ca22-46b9-9161-623da8448395 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:27.683784 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c23e34-ca22-46b9-9161-623da8448395 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:27.685654 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-03c23e34-ca22-46b9-9161-623da8448395 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:27.770944 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c23e34-ca22-46b9-9161-623da8448395 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:27.772355 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c23e34-ca22-46b9-9161-623da8448395 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:27.778679 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:27.785755 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1450/2900] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 15:03:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:03:28.944673 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc01fa9-030e-4202-8974-c3103c2363b6 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:28.945178 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc01fa9-030e-4202-8974-c3103c2363b6 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:28.945366 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc01fa9-030e-4202-8974-c3103c2363b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:28.962918 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-0bc01fa9-030e-4202-8974-c3103c2363b6 None None] MFA Rules not processed for user `bf393488726e49338e70017f5cb7c4be`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:28.964158 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-0bc01fa9-030e-4202-8974-c3103c2363b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:28.971695 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1451/2901] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Jun 15 15:03:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:03:28.989553 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c23e34-ca22-46b9-9161-623da8448395 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:29.001521 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c23e34-ca22-46b9-9161-623da8448395 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:29.006585 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d4ccd-5275-4faa-8333-365d7fed80aa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '641a81b3140f47119b5c2323a97a6681', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:29.007345 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d4ccd-5275-4faa-8333-365d7fed80aa service swift] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:29.007672 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d4ccd-5275-4faa-8333-365d7fed80aa service swift] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:29.007964 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d4ccd-5275-4faa-8333-365d7fed80aa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:29.016485 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2d4ccd-5275-4faa-8333-365d7fed80aa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:29.018171 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2d4ccd-5275-4faa-8333-365d7fed80aa service swift] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:29.026650 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1451/2902] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Jun 15 15:03:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 15:03:38.957772 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=54 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:39.023424 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=6 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:58.744706 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc01fa9-030e-4202-8974-c3103c2363b6 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:58.754139 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc01fa9-030e-4202-8974-c3103c2363b6 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:58.757686 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09f9f9-b1d7-44e9-aa92-031bb6aed45d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:58.758327 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09f9f9-b1d7-44e9-aa92-031bb6aed45d service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:58.758615 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09f9f9-b1d7-44e9-aa92-031bb6aed45d service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:58.758895 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09f9f9-b1d7-44e9-aa92-031bb6aed45d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:58.768240 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c09f9f9-b1d7-44e9-aa92-031bb6aed45d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:58.769664 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c09f9f9-b1d7-44e9-aa92-031bb6aed45d service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:58.779052 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1452/2903] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 15:03:58 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:04:08.774888 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=6 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:59.553284 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d4ccd-5275-4faa-8333-365d7fed80aa service swift] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:59.555076 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-9c2d4ccd-5275-4faa-8333-365d7fed80aa service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:59.625101 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d4ccd-5275-4faa-8333-365d7fed80aa service swift] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:59.629151 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3aff43-b761-474f-82e7-a0eb4360016d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:59.629795 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3aff43-b761-474f-82e7-a0eb4360016d service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:59.630072 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3aff43-b761-474f-82e7-a0eb4360016d service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:59.630344 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3aff43-b761-474f-82e7-a0eb4360016d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:59.632119 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-9c3aff43-b761-474f-82e7-a0eb4360016d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:59.641301 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3aff43-b761-474f-82e7-a0eb4360016d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:59.642711 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3aff43-b761-474f-82e7-a0eb4360016d service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:59.651316 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1452/2904] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:05:09.648357 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=28 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:06:18.302067 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09f9f9-b1d7-44e9-aa92-031bb6aed45d service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:18.330136 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09f9f9-b1d7-44e9-aa92-031bb6aed45d service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:18.333727 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e476f1df-c4c0-4eff-972f-154504baf3f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975ae2790ac849ff88e991231671f510', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:18.334359 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e476f1df-c4c0-4eff-972f-154504baf3f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:18.334641 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e476f1df-c4c0-4eff-972f-154504baf3f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:18.334914 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e476f1df-c4c0-4eff-972f-154504baf3f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:18.336718 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e476f1df-c4c0-4eff-972f-154504baf3f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:06:18.378174 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e476f1df-c4c0-4eff-972f-154504baf3f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:18.380203 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e476f1df-c4c0-4eff-972f-154504baf3f1 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:18.391161 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1453/2905] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 15:06:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:06:18.854361 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3aff43-b761-474f-82e7-a0eb4360016d service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:18.856007 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-9c3aff43-b761-474f-82e7-a0eb4360016d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:06:18.888479 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3aff43-b761-474f-82e7-a0eb4360016d service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:18.907051 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7dccc-10ce-4cfb-b13a-2766a7d15e0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:18.907899 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7dccc-10ce-4cfb-b13a-2766a7d15e0e service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:18.908220 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7dccc-10ce-4cfb-b13a-2766a7d15e0e service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:18.908473 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7dccc-10ce-4cfb-b13a-2766a7d15e0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:18.919751 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e5e7dccc-10ce-4cfb-b13a-2766a7d15e0e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:06:18.937430 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e7dccc-10ce-4cfb-b13a-2766a7d15e0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:18.939220 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e7dccc-10ce-4cfb-b13a-2766a7d15e0e service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:18.955563 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1453/2906] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 15:06:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:06:25.268340 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e476f1df-c4c0-4eff-972f-154504baf3f1 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:25.283685 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e476f1df-c4c0-4eff-972f-154504baf3f1 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:25.290428 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578dc90-5516-48da-b565-67fda1f311d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3185bd2458c4e22830975700888e9be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:25.291297 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578dc90-5516-48da-b565-67fda1f311d1 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:25.291691 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578dc90-5516-48da-b565-67fda1f311d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:25.292110 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578dc90-5516-48da-b565-67fda1f311d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:25.294447 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d578dc90-5516-48da-b565-67fda1f311d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:06:25.310056 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578dc90-5516-48da-b565-67fda1f311d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:25.311692 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578dc90-5516-48da-b565-67fda1f311d1 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:25.323275 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1454/2907] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 15:06:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:06:26.334985 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5aebb7-25fb-4a98-b9ff-021a000a711c None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:26.335502 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5aebb7-25fb-4a98-b9ff-021a000a711c None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:26.335616 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5aebb7-25fb-4a98-b9ff-021a000a711c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:26.355073 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-ae5aebb7-25fb-4a98-b9ff-021a000a711c None None] MFA Rules not processed for user `bf393488726e49338e70017f5cb7c4be`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:06:26.356264 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ae5aebb7-25fb-4a98-b9ff-021a000a711c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:06:26.363816 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1454/2908] 10.222.0.45 () {58 vars in 967 bytes} [Wed Jun 15 15:06:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:06:26.403222 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578dc90-5516-48da-b565-67fda1f311d1 service nova] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:26.414968 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578dc90-5516-48da-b565-67fda1f311d1 service nova] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:26.419927 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5016d-9c1f-4ec9-a4d5-2cb64b5cff51 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '641a81b3140f47119b5c2323a97a6681', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:26.420553 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5016d-9c1f-4ec9-a4d5-2cb64b5cff51 service swift] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:26.420833 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5016d-9c1f-4ec9-a4d5-2cb64b5cff51 service swift] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:26.421109 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5016d-9c1f-4ec9-a4d5-2cb64b5cff51 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:26.429337 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5016d-9c1f-4ec9-a4d5-2cb64b5cff51 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:26.430899 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5016d-9c1f-4ec9-a4d5-2cb64b5cff51 service swift] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:26.439189 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1455/2909] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Jun 15 15:06:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 15:06:33.028897 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5aebb7-25fb-4a98-b9ff-021a000a711c None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:33.117156 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5aebb7-25fb-4a98-b9ff-021a000a711c None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:33.121922 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58cd864-e2d8-4223-ae15-3871ce5d901e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:33.122731 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58cd864-e2d8-4223-ae15-3871ce5d901e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:33.123030 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58cd864-e2d8-4223-ae15-3871ce5d901e admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:33.123304 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58cd864-e2d8-4223-ae15-3871ce5d901e admin admin] PATH_INFO: `/v3/users/abf3c553d53f4118a2b4bb6ba27debe5` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:33.124487 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58cd864-e2d8-4223-ae15-3871ce5d901e admin admin] RBAC: Authorizing `identity:delete_user(user_id=abf3c553d53f4118a2b4bb6ba27debe5)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:33.126387 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58cd864-e2d8-4223-ae15-3871ce5d901e admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:33.172683 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b58cd864-e2d8-4223-ae15-3871ce5d901e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abf3c553d53f4118a2b4bb6ba27debe5'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:33.173250 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b58cd864-e2d8-4223-ae15-3871ce5d901e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abf3c553d53f4118a2b4bb6ba27debe5'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:33.186049 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b58cd864-e2d8-4223-ae15-3871ce5d901e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abf3c553d53f4118a2b4bb6ba27debe5'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:33.186696 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b58cd864-e2d8-4223-ae15-3871ce5d901e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abf3c553d53f4118a2b4bb6ba27debe5'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:33.194357 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b58cd864-e2d8-4223-ae15-3871ce5d901e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abf3c553d53f4118a2b4bb6ba27debe5'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:33.207638 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1455/2910] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 15:06:33 2022] DELETE /identity/v3/users/abf3c553d53f4118a2b4bb6ba27debe5 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:06:33.397929 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5016d-9c1f-4ec9-a4d5-2cb64b5cff51 service swift] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:33.408617 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5016d-9c1f-4ec9-a4d5-2cb64b5cff51 service swift] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:33.471534 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e19bad-8bfa-4ca6-a179-c4d763bbe456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:33.472404 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e19bad-8bfa-4ca6-a179-c4d763bbe456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:33.472693 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e19bad-8bfa-4ca6-a179-c4d763bbe456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:33.472975 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e19bad-8bfa-4ca6-a179-c4d763bbe456 admin admin] PATH_INFO: `/v3/projects/c54a1f9aea1a42bfb5fc36770120eb58` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:33.474195 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e19bad-8bfa-4ca6-a179-c4d763bbe456 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c54a1f9aea1a42bfb5fc36770120eb58)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:33.476233 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e19bad-8bfa-4ca6-a179-c4d763bbe456 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:33.525591 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-50e19bad-8bfa-4ca6-a179-c4d763bbe456 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c54a1f9aea1a42bfb5fc36770120eb58', 'request_id': 'req-50e19bad-8bfa-4ca6-a179-c4d763bbe456'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:33.533312 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-50e19bad-8bfa-4ca6-a179-c4d763bbe456 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c54a1f9aea1a42bfb5fc36770120eb58', 'request_id': 'req-50e19bad-8bfa-4ca6-a179-c4d763bbe456'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:33.540060 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-50e19bad-8bfa-4ca6-a179-c4d763bbe456 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c54a1f9aea1a42bfb5fc36770120eb58', 'request_id': 'req-50e19bad-8bfa-4ca6-a179-c4d763bbe456'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:33.551998 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:06:33.565172 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-50e19bad-8bfa-4ca6-a179-c4d763bbe456 admin admin] The token cache is being invalidate because project c54a1f9aea1a42bfb5fc36770120eb58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:06:33.565865 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-50e19bad-8bfa-4ca6-a179-c4d763bbe456 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:33.567152 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-50e19bad-8bfa-4ca6-a179-c4d763bbe456 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:33.569557 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1456/2911] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 15:06:33 2022] DELETE /identity/v3/projects/c54a1f9aea1a42bfb5fc36770120eb58 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:06:33.573623 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58cd864-e2d8-4223-ae15-3871ce5d901e admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:33.576213 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-b58cd864-e2d8-4223-ae15-3871ce5d901e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:06:33.589606 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58cd864-e2d8-4223-ae15-3871ce5d901e admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:33.630772 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:06:33.651586 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42163d4a-bfdc-4400-946a-173db2e64b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:33.652382 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42163d4a-bfdc-4400-946a-173db2e64b59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:33.652670 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42163d4a-bfdc-4400-946a-173db2e64b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:33.652937 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42163d4a-bfdc-4400-946a-173db2e64b59 admin admin] PATH_INFO: `/v3/users/afd6be4d9dcb461fbed484c1271f0006` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:33.654139 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42163d4a-bfdc-4400-946a-173db2e64b59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afd6be4d9dcb461fbed484c1271f0006)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:33.656107 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42163d4a-bfdc-4400-946a-173db2e64b59 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:33.705661 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-42163d4a-bfdc-4400-946a-173db2e64b59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afd6be4d9dcb461fbed484c1271f0006'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:33.706401 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-42163d4a-bfdc-4400-946a-173db2e64b59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afd6be4d9dcb461fbed484c1271f0006'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:33.721930 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-42163d4a-bfdc-4400-946a-173db2e64b59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afd6be4d9dcb461fbed484c1271f0006'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:33.722624 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-42163d4a-bfdc-4400-946a-173db2e64b59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afd6be4d9dcb461fbed484c1271f0006'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:33.731043 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-42163d4a-bfdc-4400-946a-173db2e64b59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afd6be4d9dcb461fbed484c1271f0006'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:33.744592 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1456/2912] 10.222.0.45 () {58 vars in 1223 bytes} [Wed Jun 15 15:06:33 2022] DELETE /identity/v3/users/afd6be4d9dcb461fbed484c1271f0006 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:06:33.919685 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e19bad-8bfa-4ca6-a179-c4d763bbe456 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:33.921603 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-50e19bad-8bfa-4ca6-a179-c4d763bbe456 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:06:33.938149 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e19bad-8bfa-4ca6-a179-c4d763bbe456 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:34.003997 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4071f6e-ec6c-4df4-9438-2ae793859370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a476d9b478cd40989e9e4e02fbc699e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66e8228804bb44bd89c3894562c27b12', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:34.004860 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4071f6e-ec6c-4df4-9438-2ae793859370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:34.005148 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4071f6e-ec6c-4df4-9438-2ae793859370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:34.005427 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4071f6e-ec6c-4df4-9438-2ae793859370 admin admin] PATH_INFO: `/v3/projects/b4aec680871c4bb4b4f2a657050972dd` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:34.007269 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4071f6e-ec6c-4df4-9438-2ae793859370 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4aec680871c4bb4b4f2a657050972dd)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:34.009258 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4071f6e-ec6c-4df4-9438-2ae793859370 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:34.057422 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b4071f6e-ec6c-4df4-9438-2ae793859370 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4aec680871c4bb4b4f2a657050972dd', 'request_id': 'req-b4071f6e-ec6c-4df4-9438-2ae793859370'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:34.065121 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b4071f6e-ec6c-4df4-9438-2ae793859370 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4aec680871c4bb4b4f2a657050972dd', 'request_id': 'req-b4071f6e-ec6c-4df4-9438-2ae793859370'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:34.072271 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b4071f6e-ec6c-4df4-9438-2ae793859370 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4aec680871c4bb4b4f2a657050972dd', 'request_id': 'req-b4071f6e-ec6c-4df4-9438-2ae793859370'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:34.082570 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b4071f6e-ec6c-4df4-9438-2ae793859370 admin admin] The token cache is being invalidate because project b4aec680871c4bb4b4f2a657050972dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:06:34.082858 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b4071f6e-ec6c-4df4-9438-2ae793859370 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:34.083533 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b4071f6e-ec6c-4df4-9438-2ae793859370 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:34.085219 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1457/2913] 10.222.0.45 () {58 vars in 1232 bytes} [Wed Jun 15 15:06:33 2022] DELETE /identity/v3/projects/b4aec680871c4bb4b4f2a657050972dd => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:06:43.740767 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=34,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:06:44.079530 dv-cinder-803823-10-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:07:07.850099 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42163d4a-bfdc-4400-946a-173db2e64b59 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:07:07.851547 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-42163d4a-bfdc-4400-946a-173db2e64b59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:07:07.953305 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42163d4a-bfdc-4400-946a-173db2e64b59 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:07:08.011916 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e49065d-03ae-4a8e-ab66-c88e3cc52537 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4128111a9aa450abb2146a5d9926dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488c79cb1704e9496fc196d8aa6c104', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:07:08.012490 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49065d-03ae-4a8e-ab66-c88e3cc52537 service placement] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:07:08.012773 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49065d-03ae-4a8e-ab66-c88e3cc52537 service placement] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:07:08.013047 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49065d-03ae-4a8e-ab66-c88e3cc52537 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:07:08.014845 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1e49065d-03ae-4a8e-ab66-c88e3cc52537 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:07:08.027059 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e49065d-03ae-4a8e-ab66-c88e3cc52537 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:07:08.028734 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e49065d-03ae-4a8e-ab66-c88e3cc52537 service placement] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:07:08.039140 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1457/2914] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Jun 15 15:07:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:07:18.034761 dv-cinder-803823-10-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=67 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}